Boost deserialisation issue : Input Stream Error at runtime (c++) - c++

I've serialised a map fine
std::map<std::string, std::string> userandPass;
saveData< std::map<std::string, std::string> >("userandPassBackup.txt", userandPass); // have removed &. why is this needed. i want to pass by reference
using the function
template <typename SaveClass>
void saveData(const std::string filename, SaveClass& c)
{
// File to be written to
boost::filesystem::remove(boost::filesystem::current_path() /
filename);
boost::filesystem::path myFile = boost::filesystem::current_path() /
filename;
// Declare an output file stream ofs that writes serialises to our
//backup file.
boost::filesystem::ofstream ofs(myFile.native());
// Declare archive ta that will use our output file stream
boost::archive::text_oarchive ta(ofs);
// Write to file
ta << c;
// How many records have been archived?
std::cout << c.size() << " records from have been successfully backed
up to " << myFile << "\n";
}
Deserialising (loading) however, fails, using:
loadData< std::map<std::string, std::string> >("userandPassBackup.txt", userandPass);
where the function is:
template <typename LoadClass>
void loadData(const std::string filename, LoadClass& c)
{
// File to be written to
boost::filesystem::remove(boost::filesystem::current_path() /
filename);
boost::filesystem::path myFile = boost::filesystem::current_path() /
filename;
// Declare an output file stream ofs that writes serialises to our
//backup file.
boost::filesystem::ifstream ifs(myFile.native());
// Declare archive ta that will use our output file stream
boost::archive::text_iarchive ta(ifs);
// Write to file
ta >> c;
// How many records have been archived?
std::cout << c.size() << " records from have been successfully backed
up to " << myFile << "\n";
}
My project compiles, but when I run it, I get the following error concerning the input stream:
libc++abi.dylib: terminating with uncaught exception of type boost::archive::archive_exception: input stream error
Abort trap: 6
I don't see why this is happening. Would anyone be so kind as to point me in the right direction?
Thanks

It seems like you copypasted loadData body from saveData. You delete file that you are trying to load as a first step by calling boost::filesystem::remove.

#VTT got the biggest bug.
Here's my free code review:
you don't need boost::filesystem to std::remove(filename)
instead of currentdir / filename you should do boost::filesystem::make_absolute
that is already default behaviour
you should not use native() since you can pass the path
the argument to saveData can be const&
Do not explicitly pass template arguments: function calls do template argument deduction.
Never mind that your comments are very redundant.
Live On Coliru
#include <boost/serialization/serialization.hpp>
#include <boost/serialization/map.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <cstdio>
#include <iostream>
#include <fstream>
template <typename SaveClass>
void saveData(const std::string filename, SaveClass const& c)
{
std::remove(filename.c_str());
std::ofstream ofs(filename);
boost::archive::text_oarchive ta(ofs);
ta << c;
}
template <typename LoadClass>
void loadData(const std::string filename, LoadClass& c)
{
std::ifstream ifs(filename);
boost::archive::text_iarchive ta(ifs);
ta >> c;
}
int main() {
std::string filename = "userandPassBackup.txt";
{
std::map<std::string, std::string> userandPass {
{ "John", "pa$$w0rd" },
{ "Jane", "welcome01" } };
saveData(filename, userandPass);
std::cout << userandPass.size() << " records from have been successfully backed up to " << filename << "\n";
}
{
std::map<std::string, std::string> userandPass;
loadData(filename, userandPass);
std::cout << userandPass.size() << " records from have been successfully restored from " << filename << "\n";
}
}
Prints
2 records from have been successfully backed up to userandPassBackup.txt
2 records from have been successfully restored from userandPassBackup.txt

Related

Serializing a map<fs::path, fs::path>?

I'm using the cereal library to serialize my classes into files, but am running into trouble with std::map - specifically, maps that use std::filesystem::path.
Say I have an Object class that contains only a map<fs::path, fs::path>, as well as the required serialize function for cereal:
struct Object
{
map<fs::path, fs::path> _map;
template<class Archive>
void serialize(Archive & archive)
{
archive(_map);
}
friend ostream& operator<<(ostream& outs, const Object c)
{
for (auto const &pair: c._map)
std::cout << "{" << pair.first << ": " << pair.second << "}\n";
return outs;
}
};
In my main function, I have:
int main()
{
// create Object
cout << "Creating object..." << endl;
Object o;
fs::path a = "bye.txt";
fs::path b = "hello.txt";
o._map[a] = b;
o._map[b] = a;
cout << "Object created: " << endl;
cout << o;
// serialize
cout << "Serializing object...." << endl;
stringstream ss;
cereal::BinaryOutputArchive oarchive(ss);
oarchive(o);
cout << "Object serialized." << endl;
// write to file
cout << "Writing serialized object to file...." << endl;
ofstream file("serialized_object");
file << ss.str();
file.close();
cout << "Object written to file." << endl;
// read from file
cout << "Reading from file..." << endl;
stringstream ss2;
fs::path ins = "serialized_object";
ifstream file_stream(ins, ios::binary);
ss2 << file_stream.rdbuf();
cereal::BinaryInputArchive iarchive(ss2);
Object out;
iarchive(out);
cout << "Object read from file." << endl;
cout << out;
}
In my output, I see the error when it reads from the serialized file:
Creating object...
Object created:
{"bye.txt": "hello.txt"}
{"hello.txt": "bye.txt"}
Serializing object....
Object serialized.
Writing serialized object to file....
Object written to file.
Reading from file...
terminate called after throwing an instance of 'cereal::Exception'
what(): Failed to read 2573 bytes from input stream! Read 28
My includes are:
#include <iostream>
#include <filesystem>
#include <fstream>
#include <string.h>
#include <map>
#include "cereal/types/map.hpp"
#include "cereal/archives/binary.hpp"
#include "cereal/types/string.hpp"
And I have included the following code at the beginning in order to be able to serialize fs::path:
namespace std
{
namespace filesystem
{
template<class Archive>
void CEREAL_LOAD_MINIMAL_FUNCTION_NAME(const Archive&, path& out, const string& in)
{
out = in;
}
template<class Archive>
string CEREAL_SAVE_MINIMAL_FUNCTION_NAME(const Archive& ar, const path& p)
{
return p.string();
}
}
}
I'm sure I'm missing something obvious, as this is my first time using cereal. Does anyone have any insight as to why I'm running into this issue?
Based on the cereal documentation, you must always use the ios::binary flag when utilizing the BinaryArchive. Here is the specific section from this page in their documentation:
When using a binary archive and a file stream (std::fstream), remember to specify the binary flag (std::ios::binary) when constructing the stream. This prevents the stream from interpreting your data as ASCII characters and altering them.
The other issue is based on how Cereal guarantees that the serialization has completed. Per this link below, the output archive object should go out of scope (invoking the destructor) prior to any attempt to read the archive. They utilize the RAII approach, just like ifstream and ofstream as noted here.
You can greatly simplify your code by allowing the Cereal library perform the writes and reads internally. Here is a modified version of your main function:
int main()
{
// create Object
cout << "Creating object..." << endl;
Object o;
fs::path a = "bye.txt";
fs::path b = "hello.txt";
o._map[a] = b;
o._map[b] = a;
cout << "Object created: " << endl;
cout << o;
cout << "Serializing object...." << endl;
// scope boundary (a function call would be cleaner)
{
ofstream ofstm("serialized_object", ios::binary);
cereal::BinaryOutputArchive oarchive(ofstm);
// serialize
oarchive(o);
}
cout << "Object serialized and written." << endl;
// deserialize the object by reading from the archive
Object out;
// scope boundary (a function would be cleaner)
{
ifstream istm("serialized_object", ios::binary);
cereal::BinaryInputArchive iarchive(istm);
//deserialize
iarchive(out);
}
cout << "Object read from file." << endl;
cout << out;
}
I hope this helps. Please test this all out prior to utilization.

How to delete unnecessary new line from string while reading file?

I have created a simple file reader function which use std::map with std::tuple. The code is as follow:
#include <iostream>
#include <fstream>
#include <vector>
#include <tuple>
#include <map>
#include <algorithm>
typedef std::map<std::string, std::tuple<int, double>> Map;
Map Readfile(std::string filename)
{
std::ifstream File(filename);
if (!File.is_open())
{
std::cout << "File '" << filename << "' does not exist.";
exit(1);
}
Map data;
std::string name;
int a;
double b;
while (std::getline(File, name, '\t') >> a >> b)
{
data.insert({name, std::tuple<int, double>(a, b)});
}
return data;
}
int main()
{
std::string file = "file_read_v3.txt";
auto mt = Readfile(file);
std::for_each(mt.begin(), mt.end(), [](std::pair<std::string,std::tuple<int, double>> it) {
std::cout << it.first << "\t" << std::get<0>(it.second) << "\t" << std::get<1>(it.second) << std::endl;
});
return 0;
}
The contents of the text file I read:
Yvis 25 63.25
Wogger X Y 2 6.2
Bilbo 111 81.3
Mary 29 154.8
The data in the file is separated by tabs ('\t'). When I compile the source code, I discovered a rather strange thing on the command line: for some reason, there is a new line break after reading each line. I see this on the command line:
Bilbo 111 81.3
Mary 29 154.8
Wogger X Y 2 6.2
Yvis 25 63.25
What is the solution to this problem? And how to avoid not sorting the scanned data?

How can I determine the current size of the file opened by std::ofstream?

I have a class that has a filestream of type ofstream. The constructor opens the file in append mode and all the messages always get written at the end of the file.
I need to write into outputFile up to some fixed size say 1Mb, then I need to close, rename, and compress it, and then open a new file of the same name.
This needs to be done when a certain size of file is reached.
I tried using tellg() but after reading stuffs (and this) on internet, I understood that this is not the right approach.
As I'm new to C++, I'm trying to find out the most optimized and correct way to get the accurate current size of file opened by ofstream?
class Logger {
std::ofstream outputFile;
int curr_size;
Logger (const std::string logfile) : outputFile(FILENAME,
std::ios::app)
{
curr_size = 0;
}
};
Somewhere in the program, I'm writing data into it:
// ??? Determine the size of current file ???
if (curr_size >= MAX_FILE_SIZE) {
outputFile.close();
//Code to rename and compress file
// ...
outputFile.open(FILENAME, std::ios::app);
curr_size = 0;
}
outputFile << message << std::endl;
outputFile.flush();
fstreams can be both input and output streams. tellg() will return the input position and tellp() will tell you of the output position. tellp() will after appending to a file tell you its size.
Consider initializing your Logger like this (edit: added example for output stream operator):
#include <iostream>
#include <fstream>
class Logger {
std::string m_filename;
std::ofstream m_os;
std::ofstream::pos_type m_curr_size;
std::ofstream::pos_type m_max_size;
public:
Logger(const std::string& logfile, std::ofstream::pos_type max_size) :
m_filename(logfile),
m_os(m_filename, std::ios::app),
m_curr_size(m_os.tellp()),
m_max_size(max_size)
{}
template<typename T>
friend Logger& operator<<(Logger&, const T&);
};
template<typename T>
Logger& operator<<(Logger& log, const T& msg) {
log.m_curr_size = (log.m_os << msg << std::flush).tellp();
if(log.m_curr_size>log.m_max_size) {
log.m_os.close();
//rename & compress
log.m_os = std::ofstream(log.m_filename, std::ios::app);
log.m_curr_size = log.m_os.tellp();
}
return log;
}
int main()
{
Logger test("log", 4LL*1024*1024*1024*1024);
test << "hello " << 10 << "\n";
return 0;
}
If you use C++17 or have an experimental version of <filesystem> available, you could also use that to get the absolute file size, like this:
#include <iostream>
#include <fstream>
#include <filesystem>
namespace fs = std::filesystem;
class Logger {
fs::directory_entry m_logfile;
std::ofstream m_os;
std::uintmax_t m_max_size;
void rotate_if_needed() {
if(max_size_reached()) {
m_os.close();
//rename & compress
m_os = std::ofstream(m_logfile.path(), std::ios::app);
}
}
public:
Logger(const std::string& logfile, std::uintmax_t max_size) :
m_logfile(logfile),
m_os(m_logfile.path(), std::ios::app),
m_max_size(max_size)
{
// make sure the path is absolute in case the process
// have changed current directory when we need to rotate the log
if(m_logfile.path().is_relative())
m_logfile = fs::directory_entry(fs::absolute(m_logfile.path()));
}
std::uintmax_t size() const { return m_logfile.file_size(); }
bool max_size_reached() const { return size()>m_max_size; }
template<typename T>
friend Logger& operator<<(Logger&, const T&);
};
template<typename T>
Logger& operator<<(Logger& log, const T& msg) {
log.m_os << msg << std::flush;
log.rotate_if_needed();
return log;
}
int main()
{
Logger test("log", 4LL*1024*1024*1024*1024);
std::cout << test.size() << "\n";
test << "hello " << 10 << "\n";
std::cout << test.size() << "\n";
test << "some more " << 3.14159 << "\n";
std::cout << test.size() << "\n";
return 0;
}
I gave it a try with tellp() and it works fine for me:
#include <iostream>
#include <fstream>
using namespace std;
int main()
{
ofstream myFile("data.txt", ios_base::app);
myFile << "Hello World!" << endl;
cout << myFile.tellp() << endl;
return 0;
}
This is the output, when calling this program:
$ ./program
13
$ ./program
26
$ ./program
39

create multiple text files inside a loop

I want to create some text file in C++. For example: I will run a loop from 1 to 5 and create the following files:
1.txt
2.txt
3.txt
4.txt
5.txt
is it possible? I have made a sample code:
#include<iostream>
#include<cstdio>
#include<cstdlib>
using namespace std;
main()
{
FILE *fp;
int i;
for(i=1;i<=5;i++)
{
//fp=fopen("%d.txt","r",i); //what will go here??
}
}
I am confused about what I will write inside the loop. how can I create those files?
char i;
char fileName[] = "0.txt";
for(i='1';i<='5';i++)
{
fileName[0]=i;
fp=fopen(fileName,"r"); //what will go here??
//...
}
You can use sprintf if this is too simple for your case;
Since you tag c++, I think fstream string is the thing to use.
A simple c++ example
#include <fstream>
#include <string>
using namespace std;
int main(){
string base(".txt");
for(int i=1;i<=5;++i){
ofstream(to_string(i)+base);// to_string() need c++11
}
}
If you still don't have to_string (you don't have c++11 or your compiler just don't have this) you can use this simple version for now. (better put this in your own namespace)
#include <string>
#include <sstream>
std::string to_string(int i){
std::stringstream s;
s << i;
return s.str();
}
You can use a std::stringstream to compose the file name before passing it to the std::ofstream constructor as a std::string.
#include <iostream>
#include <fstream>
#include <string>
#include <sstream>
#include <iomanip>
int main()
{
std::cout << "How many files do you want to create? ";
int n;
std::cin >> n;
std::cout << "How many digits do you want to display? ";
int n_digits;
std::cin >> n_digits; // i.e. zeroes == 3 -> 001.txt
std::cout << "Enter a common prefix for all the files: ";
std::string prefix;
std::cin.ignore();
std::getline(std::cin, prefix); // i.e. prefix == "file" -> file001.txt
std::string ext(".txt");
for ( int i = 1; i <= n; ++i )
{ // use a stringstream to create a file names like: prefix001.txt
std::stringstream ss;
ss << prefix << std::setfill('0') << std::setw(n_digits) << i << ext;
// open the file. If not c++11 use ss.str().c_str() instead
std::ofstream file( ss.str() );
if ( !file )
{
std::cerr << "Error: failed to create file " << ss.str() << '\n';
break;
}
// write something to the newly created file
file << "This is file: " << ss.str() << "\n\nHello!\n";
if ( !file )
{
std::cerr << "Error: failed to write to file " << ss.str() << '\n';
break;
}
}
}
#include <iostream>
#include <fstream>
int main(void)
{
std::ofstream out; // you must call out.close() inside loop to be able to open another file for writting otherwise you'll get only the first one "a.txt"
std::string sFileName;
for(char c('a'); c < 'f'; c++)
{
sFileName = c;
sFileName += ".txt";
out.open(sFileName.c_str(), std::ios::out);
// std::ofstream out(sFileName.c_str(), std::ios::out); // here you are not obliged to call out.close() because the first out is not the very second and so on...
out.close(); // very important if you use the same ofstream to open another file
}
std::cout << std::endl;
return 0;
}
*** to be able to use one ostream object in opening many files you must close the precedent file to be able to open the next otherwise it fails trying creating the next one.

writing the vector map to a file in Omnetpp

I am having problem in writing the vector map to a file. I would like to know the detail value inside the wsmdata. I know that inorder to access the detail information I need to use operator overloading like “std::ostream& operator<<(std::ostream& os, map& );” in header file as well as in .cc file. But I don’t know how to use it in detail to access the vector data or output the vector data in the file. I have bee stuck in this problem for a long time. Can anybody help ?
Here is the portion of codes:
.h file:
using std::map;
typedef std::vector<WaveShortMessage*> WaveShortMessages;
std::map<long,WaveShortMessages> receivedWarningMap;
.cc file:
// add warning message to received messages storage
receivedWarningMap[wsm->getTreeId()].push_back(wsm->dup());
std::cout<<"Wsm dup() values/ receivedWarningMap="<<wsm->dup()<<endl;
std::ofstream tracefile;
tracefile.clear();
tracefile.open("traceFile1.txt", std::ios_base::app);
for (UINT i = 0; i < receivedWarningMap[wsm->getTreeId()].size(); i++)
{
std::cout << receivedWarningMap[wsm->getTreeId()][i] << std::endl;
EV<< "MyID="<<getMyID()<< "Recepient ID"<<wsm->getRecipientAddress()<<"Neighbor ID="<< wsm->getSenderAddress()<< std::endl;
}
tracefile.close();
First of all define operator << for your class WaveShortMessage, for example this way:
std::ostream & operator<<(std::ostream &os, WaveShortMessage * wsm) {
os << "Recepient ID=" << wsm->getRecipientAddress() << "; ";
os << "Neighbor ID=" << wsm->getSenderAddress() << "; ";
// and any other fields of this class
//...
return os;
}
Then use the following code to write map to text file:
// remember to add this two includes at the beginning:
// #include <fstream>
// #include <sstream>
std::ofstream logFile;
logFile.open("log.txt"); // if exists it will be overwritten
std::stringstream ss;
for (auto it = receivedWarningMap.begin(); it != receivedWarningMap.end(); ++it) {
ss << "id=" << static_cast<int>(it->first) << "; wsms=";
for (auto it2 : it->second) {
ss << it2 << "; ";
}
ss << endl;
}
logFile << ss.str();
logFile.close();