how to discard from streams? .ignore() doesnt work for this purpose, any other methods? - c++

I have a lack of understanding about streams. The idea is, to read a file to the ifstream and then working with it. Extract Data from the stream to a string, and discard the part which is now in a string from the stream. Is that possible? Or how to handle those problems?
The following method, is for inserting a file which is properly read by the ifstream. (its a text file, containing informations about "Lost" episodes, its an episodeguide. It works fine, for one element of the class episodes. Every time i instantiate a episode file, i want to check the stream of that file, discard the informations about one episode (its indicated by "****", then the next episode starts) and process the informations discarded in a string. If I create a new object of Episode I want to discard the next informations about the episodes after "****" to the next "****" and so on.
void Episode::read(ifstream& in) {
string contents((istreambuf_iterator<char>(in)), istreambuf_iterator<char>());
size_t episodeEndPos = contents.find("****");
if ( episodeEndPos == -1) {
in.ignore(numeric_limits<char>::max());
in.clear(), in.sync();
fullContent = contents;
}
else { // empty stream for next episode
in.ignore(episodeEndPos + 4);
fullContent = contents.substr(0, episodeEndPos);
}
// fill attributes
setNrHelper();
setTitelHelper();
setFlashbackHelper();
setDescriptionHelper();
}
I tried it with inFile >> words (to read the words, this is a way to get the words out of the stream) another way i was thinking about is, to use .ignore (to ignore an amount of characters in the stream). But that doesnt work as intended. Sorry for my bad english, hopefully its clear what i want to do.

If your goal is at each call of Read() to read the next episode and advance in the file, then the trick is to to use tellg() and seekg() to bookmark the position and update it:
void Episode::Read(ifstream& in) {
streampos pos = in.tellg(); // backup current position
string fullContent;
string contents((istreambuf_iterator<char>(in)), istreambuf_iterator<char>());
size_t episodeEndPos = contents.find("****");
if (episodeEndPos == -1) {
in.ignore(numeric_limits<char>::max());
in.clear(), in.sync();
fullContent = contents;
}
else { // empty stream for next episode
fullContent = contents.substr(0, episodeEndPos);
in.seekg(pos + streamoff(episodeEndPos + 4)); // position file at next episode
}
}
In this way, you can call several time your function, every call reading the next episode.
However, please note that your approach is not optimised. When you construct your contents string from a stream iterator, you load the full rest of the file in the memory, starting at the current position in the stream. So here you keep on reading and reading again big subparts of the file.
Edit: streamlined version adapted to your format
You just need to read the line, check if it's not a separator line and concatenate...
void Episode::Read(ifstream& in) {
string line;
string fullContent;
while (getline(in, line) && line !="****") {
fullContent += line + "\n";
}
cout << "DATENSATZ: " << fullContent << endl; // just to verify content
// fill attributes
//...
}

The code you got reads the entire stream in one go just to use some part of the read text to initialize an object. Imagining a gigantic file that is almost certainly a bad idea. The easier approach is to just read until the end marker is found. In an ideal world, the end marker is easily found. Based on comments it seems to be on a line of its own which would make it quite easy:
void Episode::read(std::istream& in) {
std::string text;
for (std::string line; in >> line && line != "****"; ) {
text += line + "\n";
}
fullContent = text;
}
If the separate isn't on a line of its own, you could use code like this instead:
void Episode::read(std::istream& in) {
std::string text;
for (std::istreambuf_iterator<char> it(in), end; it != end; ++it) {
text.push_back(*it);
if (*it == '*' && 4u <= text.size() && text.substr(text.size() - 4) == "****") {
break;
}
if (4u <= text.size() && text.substr(text.size() - 4u) == "****") {
text.resize(text.size() - 4u);
}
fullContent = text;
}
Both of these approaches would simple read the file from start to end and consume the characters to be extracted in the process, stopping as soon as reading of one record is done.

Related

Storing the last value of a file from SD card using arduino

I am making a device that moves back and fourth and needs to store its last position so that upon power up, the last stored value can be grabbed from the last line of the file on an SD card, and it can resume operation. This file will then be destroyed and re-written. For this particular application homing and other methods can not be used because it must start in the spot it last was. Due to position tracking via encoder, there is no positional memory otherwise.The file is setup to be a single data column seperated by commas.
Currently I am successfully writing to the SD card as position changes, and reading the entire file to be printed on the Serial monitor. However, I really only need the last value. The length of the file will always be different do to system operation.
I have read a lot of different solutions but none of them seem to work for my application.
I can read the entire file using:
void read_file() {
// open the file for reading:
myFile = SD.open("test8.txt");
if (myFile) {
Serial.println("test8.txt:");
// read from the file until there's nothing else in it:
// read from the file until there's nothing else in it:
while (myFile.available()) {
String a = "";
for (int i = 0; i < 9; ++i)
{
int j;
char temp = myFile.read();
if (temp != ',' && temp != '\r')
{ //a=temp;
a += temp;
}
else if (temp == ',' || temp == '\r') {
j = a.toInt();
// Serial.println(a);
Serial.println(j);
break;
}
}
}
// close the file:
myFile.close();
} else {
// if the file didn't open, print an error:
Serial.println("error opening test8.txt");
}
}
This gives me a stream of the values separated by 0 like this:
20050
0
20071
0
20092
0
20113
0
20133
0
Ideally I just need 20133 to be grabbed and stored as an int.
I have also tried:
void read_file_3() {
// open the file for reading:
myFile = SD.open("test8.txt");
if (myFile) {
Serial.println("test8.txt:");
// read from the file until there's nothing else in it:
Serial.println(myFile.seek(myFile.size()));
// close the file:
myFile.close();
} else {
// if the file didn't open, print an error:
Serial.println("error opening test.txt");
}
}
This only returns "1", which does not make any sense to me.
Update:
I have found a sketch that does what I want, however it is very slow due to the use of string class. Per post #6 here: https://forum.arduino.cc/index.php?topic=379209.0
This does grab the last stored value, however it takes quite awhile as the file gets bigger, and may blow up memory.
How could this be done without the string class?
void read_file() {
// open the file for reading:
myFile = SD.open("test8.txt");
if (myFile) {
while (myFile.available())
{
String line_str = myFile.readStringUntil(','); // string lavue reading from the stream - from , to , (coma to comma)
int line = line_str.toInt();
if (line != 0) // checking for the last NON-Zero value
{
line2 = line; // this really does the trick
}
// Serial.print(line2);
// delay(100);
}
Serial.print("Last line = ");
Serial.print(line2);
// close the file:
myFile.close();
// SD.remove("test3.txt");
} else {
// if the file didn't open, print an error:
Serial.println("error opening test.txt");
}
}
Any help would be greatly appreciated!
seek returns true if it succesffuly goes to that position and false if it does not find anything there, like for instance if the file isn't that big. It does not give you the value at that position. That's why you see a 1, seek is returning true that it was able to go to the position (myFile.size()) and that's what you're printing.
Beyond that, you don't want to go to the end of the file, that would be after your number. You want to go to a position 5 characters before the end of the file if your number is 5 digits long.
Either way, once you seek that position, then you still need to use read just like you did in your first code to actually read the number. seek doesn't do that, it just takes you to that position in the file.
EDIT: Since you edited the post, I'll edit the answer to go along. You're going backwards. You had it right the first time. Use the same read method you started with, just seek the end of the file before you start reading so you don't have to read all the way through. You almost had it. The only thing you did wrong the first time was printing what you got back from seek instead of seeking the right position and then reading the file.
That thing you looked up with the String class is going backward from where you were. Forget you ever saw that. It's doing the same thing you were already doing in the first place only it's also wasting a lot of memory and code space in the process.
Use your original code and just add a seek to skip to the end of the file.
This assumes that it's always a 5 digit number. If not then you may need a little bit of tweaking:
void read_file() {
// open the file for reading:
myFile = SD.open("test8.txt");
if (myFile) {
Serial.println("test8.txt:");
/// ADDED THIS ONE LINE TO SKIP MOST OF THE FILE************
myFile.seek(myFile.size() - 5);
// read from the file until there's nothing else in it:
// read from the file until there's nothing else in it:
while (myFile.available()) {
String a = "";
for (int i = 0; i < 9; ++i)
{
int j;
char temp = myFile.read();
if (temp != ',' && temp != '\r')
{ //a=temp;
a += temp;
}
else if (temp == ',' || temp == '\r') {
j = a.toInt();
// Serial.println(a);
Serial.println(j);
break;
}
}
}
// close the file:
myFile.close();
} else {
// if the file didn't open, print an error:
Serial.println("error opening test8.txt");
}
}
See, all I've done is take your original function and add a line to seek the end to it.

How to read a complex input with istream&, string& and getline in c++?

I am very new to C++, so I apologize if this isn't a good question but I really need help in understanding how to use istream.
There is a project I have to create where it takes several amounts of input that can be on one line or multiple and then pass it to a vector (this is only part of the project and I would like to try the rest on my own), for example if I were to input this...
>> aaa bb
>> ccccc
>> ddd fff eeeee
Makes a vector of strings with "aaa", "bb", "ccccc", "ddd", "fff", "eeeee"
The input can be a char or string and the program stops asking for input when the return key is hit.
I know getline() gets a line of input and I could probably use a while loop to try and get the input such as...(correct me if I'm wrong)
while(!string.empty())
getline(cin, string);
However, I don't truly understand istream and it doesn't help that my class has not gone over pointers so I don't know how to use istream& or string& and pass it into a vector. On the project description, it said to NOT use stringstream but use functionality from getline(istream&, string&). Can anyone give somewhat of a detailed explanation as to how to make a function using getline(istream&, string&) and then how to use it in the main function?
Any little bit helps!
You're on the right way already; solely, you'd have to pre-fill the string with some dummy to enter the while loop at all. More elegant:
std::string line;
do
{
std::getline(std::cin, line);
}
while(!line.empty());
This should already do the trick reading line by line (but possibly multiple words on one line!) and exiting, if the user enters an empty line (be aware that whitespace followed by newline won't be recognised as such!).
However, if anything on the stream goes wrong, you'll be trapped in an endless loop processing previous input again and again. So best check the stream state as well:
if(!std::getline(std::cin, line))
{
// this is some sample error handling - do whatever you consider appropriate...
std::cerr << "error reading from console" << std::endl;
return -1;
}
As there might be multiple words on a single line, you'd yet have to split them. There are several ways to do so, quite an easy one is using an std::istringstream – you'll discover that it ressembles to what you likely are used to using std::cin:
std::istringstream s(line);
std::string word;
while(s >> word)
{
// append to vector...
}
Be aware that using operator>> ignores leading whitespace and stops after first trailing one (or end of stream, if reached), so you don't have to deal with explicitly.
OK, you're not allowed to use std::stringstream (well, I used std::istringstream, but I suppose this little difference doesn't count, does it?). Changes matter a little, it gets more complex, on the other hand, we can decide ourselves what counts as words an what as separators... We might consider punctuation marks as separators just like whitespace, but allow digits to be part of words, so we'd accept e. g. ab.7c d as "ab", "7c", "d":
auto begin = line.begin();
auto end = begin;
while(end != line.end()) // iterate over each character
{
if(std::isalnum(static_cast<unsigned char>(*end)))
{
// we are inside a word; don't touch begin to remember where
// the word started
++end;
}
else
{
// non-alpha-numeric character!
if(end != begin)
{
// we discovered a word already
// (i. e. we did not move begin together with end)
words.emplace_back(begin, end);
// ('words' being your std::vector<std::string> to place the input into)
}
++end;
begin = end; // skip whatever we had already
}
}
// corner case: a line might end with a word NOT followed by whitespace
// this isn't covered within the loop, so we need to add another check:
if(end != begin)
{
words.emplace_back(begin, end);
}
It shouldn't be too difficult to adjust to different interpretations of what is a separator and what counts as word (e. g. std::isalpha(...) || *end == '_' to detect underscore as part of words, but digits not). There are quite a few helper functions you might find useful...
You could input the value of the first column, then call functions based on the value:
void Process_Value_1(std::istream& input, std::string& value);
void Process_Value_2(std::istream& input, std::string& value);
int main()
{
// ...
std::string first_value;
while (input_file >> first_value)
{
if (first_value == "aaa")
{
Process_Value_1(input_file, first_value);
}
else if (first_value = "ccc")
{
Process_Value_2(input_file, first_value);
}
//...
}
return 0;
}
A sample function could be:
void Process_Value_1(std::istream& input, std::string& value)
{
std::string b;
input >> b;
std::cout << value << "\t" << b << endl;
input.ignore(1000, '\n'); // Ignore until newline.
}
There are other methods to perform the process, such as using tables of function pointers and std::map.

copying after a line has been found from a file from that position till the end of that file in c++

I have a file which holds protein coordinates as well as other information preceding it. My aim is to look for a certain line called "$PARAMETERS" and then copy from there every line succeeding it till the end of the file.
How can I get that done? This is the small code I wrote part of the entire program (that someone else wrote years ago, and I took over to upgrade his code for my research):
ifstream InFile;
InFile.open (DC_InFile.c_str(), ios::in);
while ( not InFile.eof() )
{
Line = NextLine (&InFile);
if (Line.find ("#") == 0) continue; // skip lines starting with # (comments)
if (Line.length() == 0) continue; // skip empty lines
size_t pos = Line.find("$PARAMETERS");
Line.copy(Line.begin("$PARAMETERS")+pos, Line.end("$END"));
&Line.copy >> x_1 >> y_2 >> z_3;
}
Bearing in mind that I defined Line as string
I guess you need to read data between $PARAMETERS and $END, not from $PARAMETERS until end of file. If so, you can use the following code:
string str;
while (getline(InFile, str))
{
if (str.find("#") == 0)
continue;
if (str.length() == 0)
continue;
if (str.find("$PARAMETERS") == 0)
{
double x_1, y_2, z_3; // you want to read numbers, i guess
while (getline(InFile, str))
{
if (str.find("$END") == 0)
break;
stringstream stream(str);
if (stream >> x_1 >> y_2 >> z_3)
{
// Do whatever you want with x_1, y_2 and z_3
}
}
}
}
This will handle multiple sections of data; not sure if you really want this behavior.
For example:
# comment
$PARAMETERS
1 2 3
4 5 6
$END
#unrelated data
100 200 300
$PARAMETERS
7 8 9
10 11 12
$END
I'm not sure what you want on the first line of the copied file but assuming you get that straight and you haven't read beyond the current line, you can copy the tail of the fike you are reading like this:
out << InFile.rdbuf();
Here out is the std::ostream you want to send the data to.
Note, that you should not use InFile.eof() to determine whether there is more data! Instead, you should read what you want to read and then check that the read was successful. You need to check after reading because the stream cannot know what you are trying to read before you have done so.
Following up on Dietmar's answer: it sounds to me like you
should be using std::getline until you find a line which
matches your pattern. If you want that line as part of your
output, then output it, then use Dietmar's solution to copy the
rest of the file. Something like:
while ( std::getline( in, line ) && ! isStartLine( line ) ) {
}
if ( in ) { // Since you might not have found the line
out << line << '\n'; // If you want the matching line
// You can also edit it here.
out << in.rdbuf();
}
And don't put all sorts of complicated parsing information,
with continue and break, in the loop. The results are both
unreadable and unmaintainable. Factor it out into a simple
function, as above: you'll also have a better chance of getting
it right. (In your case, should you match "$PARAMETERS #
xxx", or not?) In a separate function, it's much easier to get
it right.

Parse buffered data line by line

I want to write a parser for Wavefront OBJ file format, plain text file.
Example can be seen here: people.sc.fsu.edu/~jburkardt/data/obj/diamond.obj.
Most people use old scanf to parse this format line by line, however I would prefer to load the whole file at once to reduce IO operation count. Is there a way to parse this kind of buffered data line by line?
void ObjModelConditioner::Import(Model& asset)
{
uint8_t* buffer = SyncReadFile(asset.source_file_info());
delete [] buffer;
}
Or would it be preferable to load whole file into a string and try to parse that?
After a while It seems I found sufficient (and simple) solution. Since my goal is to create asset conditioning pipeline, the code has to be able to handle large amounts of data efficiently. Data can be read into a string at once and once loaded, stringstream can be initialized with this string.
std::string data;
SyncReadFile(asset.source_file_info(), data);
std::stringstream data_stream(data);
std::string line;
Then I simply call getline():
while(std::getline(data_stream, line))
{
std::stringstream line_stream(line);
std::string type_token;
line_stream >> type_token;
if (type_token == "v") {
// Vertex position
Vector3f position;
line_stream >> position.x >> position.y >> position.z;
// ...
}
else if (type_token == "vn") {
// Vertex normal
}
else if (type_token == "vt") {
// Texture coordinates
}
else if (type_token == "f") {
// Face
}
}
Here's a function that splits a char array into a vector of strings (assuming each new string starts with '\n' symbol):
#include <iostream>
#include <vector>
std::vector< std::string >split(char * arr)
{
std::string str = arr;
std::vector< std::string >result;
int beg=0, end=0;//begining and end of each line in the array
while( end = str.find( '\n', beg + 1 ) )
{
if(end == -1)
{
result.push_back(str.substr(beg));
break;
}
result.push_back(str.substr(beg, end - beg));
beg = end;
}
return result;
}
Here's the usage:
int main()
{
char * a = "asdasdasdasdasd \n asdasdasd \n asdasd";
std::vector< std::string >result = split(a);
}
If you've got the raw data in a char[] (or a unsigned char[]), and
you know its length, it's pretty trivial to write an input only, no seek
supported streambuf which will allow you to create an std::istream
and to use std::getline on it. Just call:
setg( start, start, start + length );
in the constructor. (Nothing else is needed.)
It really depends on how you're going to parse the text. One way to do this would be simply to read the data into a vector of strings. I'll assume that you've already covered issues such as scaleability / use of memory etc.
std::vector<std::string> lines;
std::string line;
ifstream file(filename.c_str(), ios_base::in);
while ( getline( file, line ) )
{
lines.push_back( line );
}
file.close();
This would cache your file in lines. Next you need to go through lines
for ( std::vector<std::string>::const_iterator it = lines.begin();
it != lines.end(); ++it)
{
const std::string& line = *it;
if ( line.empty() )
continue;
switch ( line[0] )
{
case 'g':
// Some stuff
break;
case 'v':
// Some stuff
break;
case 'f':
// Some stuff
break;
default:
// Default stuff including '#' (probably nothing)
}
}
Naturally, this is very simplistic and depends largely on what you want to do with your file.
The size of the file that you've given as an example is hardly likely to cause IO stress (unless you're using some very lightweight equipment) but if you're reading many files at once I suppose it might be an issue.
I think your concern here is to minimise IO and I'm not sure that this solution will really help that much since you're going to be iterating over a collection twice. If you need to go back and keep reading the same file over and over again, then it will definitely speed things up to cache the file in memory but there are just as easy ways to do this such as memory mapping a file and using normal file accessing. If you're really concerned, then try profiling a solution like this against simply processing the file directly as you read from IO.

Reading multiple files

I want to alternate between reading multiple files. Below is a simplified version of my code.
ifstream* In_file1 = new ifstream("a.dat", ios::binary);
ifstream* In_file2 = new ifstream("b..dat", ios::binary);
ifstream* In_file;
int ID;
In_file = In_file1;
int ID = 0;
//LOOPING PORTION
if (In_file -> eof())
{
In_file -> seekg(0, ios_base::beg);
In_file->close();
switch (ID)
{
case 0:
In_file = In_file2; ID = 1; break;
case 1:
In_file = In_file1; ID = 0; break;
}
}
//some codes
:
:
In_file->read (data, sizeof(double));
//LOOPING PORTION
The code works well if I am reading the files one time and I thought that everything was cool. However, if the part termed 'looping portion' is within a loop, then the behaviour becomes weird and I start having a single repeating output. Please, can someone tell me what is wrong and how I can fix it? If you have a better method of tacking the problem, please suggest. I appreciate it.
//SOLVED
Thank you everybody for your comments, I appreciate it. Here is what I simple did:
Instead of the original
switch (ID)
{
case 0:
In_file = In_file2; ID = 1; break;
case 1:
In_file = In_file1; ID = 0; break;
}
I simply did
switch (ID)
{
case 0:
In_file = new ifstream("a.dat", ios::binary); ID = 1; break;
case 1:
In_file = new ifstream("b.dat", ios::binary); ID = 0; break;
}
Now it works like charm and I can loop as much as I want:-). I appreciate your comments, great to know big brother still helps.
Let's see: the code you posted works fine, and you want us to tell you
what's wrong with the code you didn't post. That's rather difficult.
Still, the code you posted probably doesn't work correctly either.
std::istream::eof can only be used reliably after an input (or some
other operation) has failed; in the code you've posted, it will almost
certainly be false, regardless.
In addition: there's no need to dynamically allocate ifstream; in
fact, there are almost no cases where dynamic allocation of ifstream
is appropriate. And you don't check that the opens have succeeded.
If you want to read two files, one after the other, the simplest way is
to use two loops, one after the other (calling a common function for
processing the data). If for some reason that's not appropriate, I'd
use a custom streambuf, which takes a list of filenames in the
constructor, and advances to the next whenever it reaches end of file on
one, only returning EOF when it has reached the end of all of the
files. (The only complication in doing this is what to do if one of the
opens fails. I do this often enough that it's part of my tool kit,
and I use a callback to handle failure. For a one time use, however,
you can just hard code in whatever is appropriate.)
As a quick example:
// We define our own streambuf, deriving from std::streambuf
// (All istream and ostream delegate to a streambuf for the
// actual data transfer; we'll use an instance of this to
// initialize the istream we're going to read from.)
class MultiFileInputStreambuf : public std::streambuf
{
// The list of files we will process
std::vector<std::string> m_filenames;
// And our current position in the list (actually
// one past the current position, since we increment
// it when we open the file).
std::vector<std::string>::const_iterator m_current;
// Rather than create a new filebuf for each file, we'll
// reuse this one, closing any previously open file, and
// opening a new file, as needed.
std::filebuf m_streambuf;
protected:
// This is part of the protocol for streambuf. The base
// class will call this function anytime it needs to
// get a character, and there aren't any in the buffer.
// This function can set up a buffer, if it wants, but
// in this case, the buffering is handled by the filebuf,
// so it's likely not worth the bother. (But this depends
// on the cost of virtual functions---without a buffer,
// each character read will require a virtual function call
// to get here.
//
// The protocol is to return the next character, or EOF if
// there isn't one.
virtual int underflow()
{
// Get one character from the current streambuf.
int result = m_streambuf.sgetc();
// As long as 1) the current streambuf is at end of file,
// and 2) there are more files to read, open the next file
// and try to get a character from it.
while ( result == EOF && m_current != m_filenames.eof() ) {
m_streambuf.close();
m_streambuf.open( m_current->c_str(), std::ios::in );
if ( !m_streambuf.is_open() )
// Error handling here...
++ m_current;
result = m_streambuf.sgetc();
}
// We've either gotten a character from the (now) current
// streambuf, or there are no more files, and we'll return
// the EOF from our last attempt at reading.
return result;
}
public:
// Use a template and two iterators to initialize the list
// of files from any STL sequence whose elements can be
// implicitly converted to std::string.
template<typename ForwardIterator>
MultiFileInputStreambuf(ForwardIterator begin, ForwardIterator end)
: m_filenames(begin, end)
, m_current(m_filenames.begin())
{
}
};
#include <iostream>
#include <fstream>
#include <string>
#define NO_OF_FILES 2
int main ()
{
std::ifstream in;
std::string line;
std::string files[NO_OF_FILES] =
{
"file1.txt",
"file2.txt",
};
// start our engine!
for (int i = 0; i < NO_OF_FILES; i++)
{
in.open(files[i].c_str(), std::fstream::in);
if (in.is_open())
{
std::cout << "reading... " << files[i] << endl;
while (in.good())
{
getline(in, line);
std::cout << line << std::endl;
}
in.close();
std::cout << "SUCCESS" << std::endl;
}
else
std::cout << "Error: unable to open " + files[i] << std::endl;
}
return 0;
}