I am in the process of adding the ability to get data over the network to code that used to only read local files. The network library I am using sends and receives data in the form of a vector<uint8_t>. I'd like to be able to reuse the code that processes the data after reading the file, but that code expects a std::istream, is there a way to have an istream read the vector data? It's the same data so I feel like there should be a way, but i haven't been able to find or figure out code for how to do it.
current code:
std::ifstream stream("data.img", std::ios::in | std::ios::binary | std::ios::ate);
if (!stream.is_open())
{
throw std::invalid_argument("Could not open file.");
}
// the arg for processData is std::istream
processData(stream);
network framework:
vector<uint8_t> data = networkMessage.data;
// need some way to create istream from data
std::istream stream = ?
processData(stream);
stream.close();
Is there a way to do this, or am I barking up the wrong tree?
std::basic_istream gets its data from an associated std::basic_streambuf derived class. The STL provides such classes for file I/O and string I/O, but not for memory I/O or network I/O.
You could easily write (or find a 3rd party) memory-based streambuf class that uses the std::vector as its underlying buffer, and then you can construct an std::istream that uses that memory streambuf. For example (using the imemstream class from
this answer):
std::vector<uint8_t> &data = networkMessage.data;
imemstream stream(reinterpret_cast<const char*>(data.data()), data.size());
processData(stream);
Well C++ does actually have a class for this - istrstream, and you could use it like this:
vector<uint8_t> data = ...;
// need some way to create istream from data
std::istrstream stream(reinterpret_cast<const char*>(data.data()), data.size());
processData(stream);
As far as I can tell this doesn't copy the data, unlike the other answers. However it was also deprecated in C++98 because it's hard to know who is responsible for freeing the buffer, so you may want to write your own equivalent.
The istream is a reference of raw data. It doesn't hold the data, and just a visitor, by keeping some char* pointers of the begin and end of data memory address.
The storage in vector<> is continuous, but by using push_back(), the storage address may changed, (copied inner vector)
So it's possible to make an istream to const vector
The reference
https://en.cppreference.com/w/cpp/io/basic_istream
https://www.cplusplus.com/reference/streambuf/streambuf/
The shortest example
class vectorbuf : public std::streambuf {
public:
vectorbuf(std::vector<uint8_t> &v){
setg((char*)v.data(), (char*)v.data(), (char*)(v.data() + v.size()));
}
~vectorbuf() {}
};
//Usage:
vector<uint8_t> arr{11,12,13,14,15,16};
vectorbuf vbuf(arr);
std::istream is(&vbuf);
The full WRONG sample code
#include <streambuf>
#include <iostream>
#include <iomanip>
#include <vector>
using namespace std;
template<typename T>
class vectorbuf : public std::streambuf {
public:
vectorbuf(std::vector<T> &v) : _value(v) {
char *bptr = (char*)_value.data();
char *eptr = (char*)(_value.data() + _value.size());
setg(bptr, bptr, eptr);
cout<<"Setg: "<<(void*)bptr<<" "<<(void*)eptr<<endl;
}
~vectorbuf() {}
//Zone start ---
//Note: this zone of code can be commented since the virtual function in base class do same
protected:
virtual int underflow() {
char *bptr = (char*)_value.data();
char *new_eptr = (char*)(_value.data() + _value.size());
cout<<"[underflow() when gptr()="<<(void*)gptr()
<<", now_bptr="<<(void*)bptr<<" now_eptr="<<(void*)new_eptr<<"]";
return traits_type::eof();
//since the vector& must not modified, the code below is unnecessary.
if (new_eptr == egptr())
return traits_type::eof();
setg(bptr, gptr(), new_eptr);
return *gptr();
}
//Zone end ---
private:
std::vector<T> &_value;
};
int main() {
vector<int> arr{'a',12,13,14,15};
cout<<"The array: ";
for (int i=0; i<arr.size(); i++)
cout<<arr[i]<<" ";
cout<<endl;
cout<<" storage: ";
for (int i=0; i<arr.size()*sizeof(int); i++) {
char *ptr = (char*)arr.data();
cout<<static_cast<int>(ptr[i])<<" ";
}
cout<<endl;
vectorbuf<int> vbuf(arr);
std::istream is(&vbuf);
arr.push_back(16); //!!! wrong code here !!!
//the size of arr is 6*4 == 24, with sizeof(int)==4
for (int i=0; i<26; i++) {
cout<<"good?"<<is.good()
<<", fail?"<<is.fail()
<<", bad?"<<is.bad()
<<", eof?"<<is.eof()
<<", tellg="<<is.tellg();
//Note there must be char
//'int a' would not accepted and make is.fail() to true
//and std::noskipws is also importanted
char a;
is>>std::noskipws>>a;
int out = a;
cout<<", Read from arr: "<<out<<endl;
}
return 0;
}
You can do this via assigning the data to a std::string and use a std::istringstream bound to that (leaving aside the unsigned char to signed char conversion issues):
std::string s((char*)networkMessage.data(),networkMessage.size());
std::istringstream iss(s);
std::istream& stream = iss;
// ^ Note the reference here.
processData(stream);
stream.close();
This will work with vector of any type, not just uint8_t:
std
template <class T>
auto make_istringstream_std_1(const std::vector<T>& v) -> std::istringstream
{
using namespace std::string_literals;
std::string str;
for (auto& e : v)
{
str += std::to_string(e) + " "s;
}
// the trailing space is not an issue
return std::istringstream{str};
}
std algorithm
template <class T>
auto make_istringstream_std_2(const std::vector<T>& v) -> std::istringstream
{
std::stringstream ss;
std::copy(v.begin(), v.end(), std::ostream_iterator<int>(ss, " "));
// the trailing space is not an issue
return std::istringstream{ss.str()};
}
boost
template <class T>
auto make_istringstream_boost(const std::vector<T>& v) -> std::istringstream
{
using boost::adaptors::transformed;
using boost::algorithm::join;
return std::istringstream{
join(v | transformed([](int a) { return std::to_string(a); }), " ")};
}
attribution:
How to transform a vector<int> into a string?
A good example for boost::algorithm::join
Related
I'm not a c++ expert but I've serialized things a couple of times in the past. Unfortunately this time I'm trying to serialize a class which contains an std::string, which I understand is pretty much like serializing a pointer.
I can write out the class to a file and read it back in again. All int fields are fine, but the std::string field gives an "address out of bounds" error, presumably because it points to data which is no longer there.
Is there a standard workaround for this? I don't want to go back to char arrays, but at least I know they work in this situation. I can provide code if necessary, but I'm hoping I've explained my problem well.
I'm serializing by casting the class to a char* and writing it to a file with std::fstream. Reading of course is just the reverse.
I'm serializing by casting the class to a char* and writing it to a
file with fstream. Reading of course is just the reverse.
Unfortunately, this only works as long as there are no pointers involved. You might want to give your classes void MyClass::serialize(std::ostream) and void MyClass::deserialize(std::ifstream), and call those. For this case, you'd want
std::ostream& MyClass::serialize(std::ostream &out) const {
out << height;
out << ',' //number seperator
out << width;
out << ',' //number seperator
out << name.size(); //serialize size of string
out << ',' //number seperator
out << name; //serialize characters of string
return out;
}
std::istream& MyClass::deserialize(std::istream &in) {
if (in) {
int len=0;
char comma;
in >> height;
in >> comma; //read in the seperator
in >> width;
in >> comma; //read in the seperator
in >> len; //deserialize size of string
in >> comma; //read in the seperator
if (in && len) {
std::vector<char> tmp(len);
in.read(tmp.data() , len); //deserialize characters of string
name.assign(tmp.data(), len);
}
}
return in;
}
You may also want to overload the stream operators for easier use.
std::ostream &operator<<(std::ostream& out, const MyClass &obj)
{obj.serialize(out); return out;}
std::istream &operator>>(std::istream& in, MyClass &obj)
{obj.deserialize(in); return in;}
Simply writing the binary contents of an object into a file is not only unportable but, as you've recognized, doesn't work for pointer data. You basically have two options: either you write a real serialization library, which handles std::strings properly by e.g. using c_str() to output the actual string to the file, or you use the excellent boost serialization library. If at all possible, I'd recommend the latter, you can then serialize with a simple code like this:
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/string.hpp>
class A {
private:
std::string s;
public:
template<class Archive>
void serialize(Archive& ar, const unsigned int version)
{
ar & s;
}
};
Here, the function serialize works for serializing and deserializing the data, depending on how you call it. See the documentation for more information.
The easiest serialization method for strings or other blobs with variable size is to serialize first the size as you serialize integers, then just copy the content to the output stream.
When reading you first read the size, then allocate the string and then fill it by reading the correct number of bytes from the stream.
An alternative is to use a delimiter and escaping, but requires more code and is slower both on serialization and deserialization (however the result can be kept human readable).
You'll have to use a more complicated method of serialization than casting a class to a char* and writing it to a file if your class contains any exogenous data (string does). And you're correct about why you're getting a segmentation fault.
I would make a member function that would take an fstream and read in the data from it as well as an inverse function which would take an fstream and write it's contents to it to be restored later, like this:
class MyClass {
pubic:
MyClass() : str() { }
void serialize(ostream& out) {
out << str;
}
void restore(istream& in) {
in >> str;
}
string& data() const { return str; }
private:
string str;
};
MyClass c;
c.serialize(output);
// later
c.restore(input);
You can also define operator<< and operator>> to work with istream and ostream to serialize and restore your class as well if you want that syntactic sugar.
Why not just something along the lines of:
std::ofstream ofs;
...
ofs << my_str;
and then:
std::ifstream ifs;
...
ifs >> my_str;
/*!
* reads binary data into the string.
* #status : OK.
*/
class UReadBinaryString
{
static std::string read(std::istream &is, uint32_t size)
{
std::string returnStr;
if(size > 0)
{
CWrapPtr<char> buff(new char[size]); // custom smart pointer
is.read(reinterpret_cast<char*>(buff.m_obj), size);
returnStr.assign(buff.m_obj, size);
}
return returnStr;
}
};
class objHeader
{
public:
std::string m_ID;
// serialize
std::ostream &operator << (std::ostream &os)
{
uint32_t size = (m_ID.length());
os.write(reinterpret_cast<char*>(&size), sizeof(uint32_t));
os.write(m_ID.c_str(), size);
return os;
}
// de-serialize
std::istream &operator >> (std::istream &is)
{
uint32_t size;
is.read(reinterpret_cast<char*>(&size), sizeof(uint32_t));
m_ID = UReadBinaryString::read(is, size);
return is;
}
};
I haven't coded C++ in a long time, but perhaps you could serialize an array of char.
Then, when you open your file, your string would just point to the array.
Just an idea.
I'm not a c++ expert but I've serialized things a couple of times in the past. Unfortunately this time I'm trying to serialize a class which contains an std::string, which I understand is pretty much like serializing a pointer.
I can write out the class to a file and read it back in again. All int fields are fine, but the std::string field gives an "address out of bounds" error, presumably because it points to data which is no longer there.
Is there a standard workaround for this? I don't want to go back to char arrays, but at least I know they work in this situation. I can provide code if necessary, but I'm hoping I've explained my problem well.
I'm serializing by casting the class to a char* and writing it to a file with std::fstream. Reading of course is just the reverse.
I'm serializing by casting the class to a char* and writing it to a
file with fstream. Reading of course is just the reverse.
Unfortunately, this only works as long as there are no pointers involved. You might want to give your classes void MyClass::serialize(std::ostream) and void MyClass::deserialize(std::ifstream), and call those. For this case, you'd want
std::ostream& MyClass::serialize(std::ostream &out) const {
out << height;
out << ',' //number seperator
out << width;
out << ',' //number seperator
out << name.size(); //serialize size of string
out << ',' //number seperator
out << name; //serialize characters of string
return out;
}
std::istream& MyClass::deserialize(std::istream &in) {
if (in) {
int len=0;
char comma;
in >> height;
in >> comma; //read in the seperator
in >> width;
in >> comma; //read in the seperator
in >> len; //deserialize size of string
in >> comma; //read in the seperator
if (in && len) {
std::vector<char> tmp(len);
in.read(tmp.data() , len); //deserialize characters of string
name.assign(tmp.data(), len);
}
}
return in;
}
You may also want to overload the stream operators for easier use.
std::ostream &operator<<(std::ostream& out, const MyClass &obj)
{obj.serialize(out); return out;}
std::istream &operator>>(std::istream& in, MyClass &obj)
{obj.deserialize(in); return in;}
Simply writing the binary contents of an object into a file is not only unportable but, as you've recognized, doesn't work for pointer data. You basically have two options: either you write a real serialization library, which handles std::strings properly by e.g. using c_str() to output the actual string to the file, or you use the excellent boost serialization library. If at all possible, I'd recommend the latter, you can then serialize with a simple code like this:
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/string.hpp>
class A {
private:
std::string s;
public:
template<class Archive>
void serialize(Archive& ar, const unsigned int version)
{
ar & s;
}
};
Here, the function serialize works for serializing and deserializing the data, depending on how you call it. See the documentation for more information.
The easiest serialization method for strings or other blobs with variable size is to serialize first the size as you serialize integers, then just copy the content to the output stream.
When reading you first read the size, then allocate the string and then fill it by reading the correct number of bytes from the stream.
An alternative is to use a delimiter and escaping, but requires more code and is slower both on serialization and deserialization (however the result can be kept human readable).
You'll have to use a more complicated method of serialization than casting a class to a char* and writing it to a file if your class contains any exogenous data (string does). And you're correct about why you're getting a segmentation fault.
I would make a member function that would take an fstream and read in the data from it as well as an inverse function which would take an fstream and write it's contents to it to be restored later, like this:
class MyClass {
pubic:
MyClass() : str() { }
void serialize(ostream& out) {
out << str;
}
void restore(istream& in) {
in >> str;
}
string& data() const { return str; }
private:
string str;
};
MyClass c;
c.serialize(output);
// later
c.restore(input);
You can also define operator<< and operator>> to work with istream and ostream to serialize and restore your class as well if you want that syntactic sugar.
Why not just something along the lines of:
std::ofstream ofs;
...
ofs << my_str;
and then:
std::ifstream ifs;
...
ifs >> my_str;
/*!
* reads binary data into the string.
* #status : OK.
*/
class UReadBinaryString
{
static std::string read(std::istream &is, uint32_t size)
{
std::string returnStr;
if(size > 0)
{
CWrapPtr<char> buff(new char[size]); // custom smart pointer
is.read(reinterpret_cast<char*>(buff.m_obj), size);
returnStr.assign(buff.m_obj, size);
}
return returnStr;
}
};
class objHeader
{
public:
std::string m_ID;
// serialize
std::ostream &operator << (std::ostream &os)
{
uint32_t size = (m_ID.length());
os.write(reinterpret_cast<char*>(&size), sizeof(uint32_t));
os.write(m_ID.c_str(), size);
return os;
}
// de-serialize
std::istream &operator >> (std::istream &is)
{
uint32_t size;
is.read(reinterpret_cast<char*>(&size), sizeof(uint32_t));
m_ID = UReadBinaryString::read(is, size);
return is;
}
};
I haven't coded C++ in a long time, but perhaps you could serialize an array of char.
Then, when you open your file, your string would just point to the array.
Just an idea.
I'm not a c++ expert but I've serialized things a couple of times in the past. Unfortunately this time I'm trying to serialize a class which contains an std::string, which I understand is pretty much like serializing a pointer.
I can write out the class to a file and read it back in again. All int fields are fine, but the std::string field gives an "address out of bounds" error, presumably because it points to data which is no longer there.
Is there a standard workaround for this? I don't want to go back to char arrays, but at least I know they work in this situation. I can provide code if necessary, but I'm hoping I've explained my problem well.
I'm serializing by casting the class to a char* and writing it to a file with std::fstream. Reading of course is just the reverse.
I'm serializing by casting the class to a char* and writing it to a
file with fstream. Reading of course is just the reverse.
Unfortunately, this only works as long as there are no pointers involved. You might want to give your classes void MyClass::serialize(std::ostream) and void MyClass::deserialize(std::ifstream), and call those. For this case, you'd want
std::ostream& MyClass::serialize(std::ostream &out) const {
out << height;
out << ',' //number seperator
out << width;
out << ',' //number seperator
out << name.size(); //serialize size of string
out << ',' //number seperator
out << name; //serialize characters of string
return out;
}
std::istream& MyClass::deserialize(std::istream &in) {
if (in) {
int len=0;
char comma;
in >> height;
in >> comma; //read in the seperator
in >> width;
in >> comma; //read in the seperator
in >> len; //deserialize size of string
in >> comma; //read in the seperator
if (in && len) {
std::vector<char> tmp(len);
in.read(tmp.data() , len); //deserialize characters of string
name.assign(tmp.data(), len);
}
}
return in;
}
You may also want to overload the stream operators for easier use.
std::ostream &operator<<(std::ostream& out, const MyClass &obj)
{obj.serialize(out); return out;}
std::istream &operator>>(std::istream& in, MyClass &obj)
{obj.deserialize(in); return in;}
Simply writing the binary contents of an object into a file is not only unportable but, as you've recognized, doesn't work for pointer data. You basically have two options: either you write a real serialization library, which handles std::strings properly by e.g. using c_str() to output the actual string to the file, or you use the excellent boost serialization library. If at all possible, I'd recommend the latter, you can then serialize with a simple code like this:
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/string.hpp>
class A {
private:
std::string s;
public:
template<class Archive>
void serialize(Archive& ar, const unsigned int version)
{
ar & s;
}
};
Here, the function serialize works for serializing and deserializing the data, depending on how you call it. See the documentation for more information.
The easiest serialization method for strings or other blobs with variable size is to serialize first the size as you serialize integers, then just copy the content to the output stream.
When reading you first read the size, then allocate the string and then fill it by reading the correct number of bytes from the stream.
An alternative is to use a delimiter and escaping, but requires more code and is slower both on serialization and deserialization (however the result can be kept human readable).
You'll have to use a more complicated method of serialization than casting a class to a char* and writing it to a file if your class contains any exogenous data (string does). And you're correct about why you're getting a segmentation fault.
I would make a member function that would take an fstream and read in the data from it as well as an inverse function which would take an fstream and write it's contents to it to be restored later, like this:
class MyClass {
pubic:
MyClass() : str() { }
void serialize(ostream& out) {
out << str;
}
void restore(istream& in) {
in >> str;
}
string& data() const { return str; }
private:
string str;
};
MyClass c;
c.serialize(output);
// later
c.restore(input);
You can also define operator<< and operator>> to work with istream and ostream to serialize and restore your class as well if you want that syntactic sugar.
Why not just something along the lines of:
std::ofstream ofs;
...
ofs << my_str;
and then:
std::ifstream ifs;
...
ifs >> my_str;
/*!
* reads binary data into the string.
* #status : OK.
*/
class UReadBinaryString
{
static std::string read(std::istream &is, uint32_t size)
{
std::string returnStr;
if(size > 0)
{
CWrapPtr<char> buff(new char[size]); // custom smart pointer
is.read(reinterpret_cast<char*>(buff.m_obj), size);
returnStr.assign(buff.m_obj, size);
}
return returnStr;
}
};
class objHeader
{
public:
std::string m_ID;
// serialize
std::ostream &operator << (std::ostream &os)
{
uint32_t size = (m_ID.length());
os.write(reinterpret_cast<char*>(&size), sizeof(uint32_t));
os.write(m_ID.c_str(), size);
return os;
}
// de-serialize
std::istream &operator >> (std::istream &is)
{
uint32_t size;
is.read(reinterpret_cast<char*>(&size), sizeof(uint32_t));
m_ID = UReadBinaryString::read(is, size);
return is;
}
};
I haven't coded C++ in a long time, but perhaps you could serialize an array of char.
Then, when you open your file, your string would just point to the array.
Just an idea.
I'm not a c++ expert but I've serialized things a couple of times in the past. Unfortunately this time I'm trying to serialize a class which contains an std::string, which I understand is pretty much like serializing a pointer.
I can write out the class to a file and read it back in again. All int fields are fine, but the std::string field gives an "address out of bounds" error, presumably because it points to data which is no longer there.
Is there a standard workaround for this? I don't want to go back to char arrays, but at least I know they work in this situation. I can provide code if necessary, but I'm hoping I've explained my problem well.
I'm serializing by casting the class to a char* and writing it to a file with std::fstream. Reading of course is just the reverse.
I'm serializing by casting the class to a char* and writing it to a
file with fstream. Reading of course is just the reverse.
Unfortunately, this only works as long as there are no pointers involved. You might want to give your classes void MyClass::serialize(std::ostream) and void MyClass::deserialize(std::ifstream), and call those. For this case, you'd want
std::ostream& MyClass::serialize(std::ostream &out) const {
out << height;
out << ',' //number seperator
out << width;
out << ',' //number seperator
out << name.size(); //serialize size of string
out << ',' //number seperator
out << name; //serialize characters of string
return out;
}
std::istream& MyClass::deserialize(std::istream &in) {
if (in) {
int len=0;
char comma;
in >> height;
in >> comma; //read in the seperator
in >> width;
in >> comma; //read in the seperator
in >> len; //deserialize size of string
in >> comma; //read in the seperator
if (in && len) {
std::vector<char> tmp(len);
in.read(tmp.data() , len); //deserialize characters of string
name.assign(tmp.data(), len);
}
}
return in;
}
You may also want to overload the stream operators for easier use.
std::ostream &operator<<(std::ostream& out, const MyClass &obj)
{obj.serialize(out); return out;}
std::istream &operator>>(std::istream& in, MyClass &obj)
{obj.deserialize(in); return in;}
Simply writing the binary contents of an object into a file is not only unportable but, as you've recognized, doesn't work for pointer data. You basically have two options: either you write a real serialization library, which handles std::strings properly by e.g. using c_str() to output the actual string to the file, or you use the excellent boost serialization library. If at all possible, I'd recommend the latter, you can then serialize with a simple code like this:
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/string.hpp>
class A {
private:
std::string s;
public:
template<class Archive>
void serialize(Archive& ar, const unsigned int version)
{
ar & s;
}
};
Here, the function serialize works for serializing and deserializing the data, depending on how you call it. See the documentation for more information.
The easiest serialization method for strings or other blobs with variable size is to serialize first the size as you serialize integers, then just copy the content to the output stream.
When reading you first read the size, then allocate the string and then fill it by reading the correct number of bytes from the stream.
An alternative is to use a delimiter and escaping, but requires more code and is slower both on serialization and deserialization (however the result can be kept human readable).
You'll have to use a more complicated method of serialization than casting a class to a char* and writing it to a file if your class contains any exogenous data (string does). And you're correct about why you're getting a segmentation fault.
I would make a member function that would take an fstream and read in the data from it as well as an inverse function which would take an fstream and write it's contents to it to be restored later, like this:
class MyClass {
pubic:
MyClass() : str() { }
void serialize(ostream& out) {
out << str;
}
void restore(istream& in) {
in >> str;
}
string& data() const { return str; }
private:
string str;
};
MyClass c;
c.serialize(output);
// later
c.restore(input);
You can also define operator<< and operator>> to work with istream and ostream to serialize and restore your class as well if you want that syntactic sugar.
Why not just something along the lines of:
std::ofstream ofs;
...
ofs << my_str;
and then:
std::ifstream ifs;
...
ifs >> my_str;
/*!
* reads binary data into the string.
* #status : OK.
*/
class UReadBinaryString
{
static std::string read(std::istream &is, uint32_t size)
{
std::string returnStr;
if(size > 0)
{
CWrapPtr<char> buff(new char[size]); // custom smart pointer
is.read(reinterpret_cast<char*>(buff.m_obj), size);
returnStr.assign(buff.m_obj, size);
}
return returnStr;
}
};
class objHeader
{
public:
std::string m_ID;
// serialize
std::ostream &operator << (std::ostream &os)
{
uint32_t size = (m_ID.length());
os.write(reinterpret_cast<char*>(&size), sizeof(uint32_t));
os.write(m_ID.c_str(), size);
return os;
}
// de-serialize
std::istream &operator >> (std::istream &is)
{
uint32_t size;
is.read(reinterpret_cast<char*>(&size), sizeof(uint32_t));
m_ID = UReadBinaryString::read(is, size);
return is;
}
};
I haven't coded C++ in a long time, but perhaps you could serialize an array of char.
Then, when you open your file, your string would just point to the array.
Just an idea.
I'm not a c++ expert but I've serialized things a couple of times in the past. Unfortunately this time I'm trying to serialize a class which contains an std::string, which I understand is pretty much like serializing a pointer.
I can write out the class to a file and read it back in again. All int fields are fine, but the std::string field gives an "address out of bounds" error, presumably because it points to data which is no longer there.
Is there a standard workaround for this? I don't want to go back to char arrays, but at least I know they work in this situation. I can provide code if necessary, but I'm hoping I've explained my problem well.
I'm serializing by casting the class to a char* and writing it to a file with std::fstream. Reading of course is just the reverse.
I'm serializing by casting the class to a char* and writing it to a
file with fstream. Reading of course is just the reverse.
Unfortunately, this only works as long as there are no pointers involved. You might want to give your classes void MyClass::serialize(std::ostream) and void MyClass::deserialize(std::ifstream), and call those. For this case, you'd want
std::ostream& MyClass::serialize(std::ostream &out) const {
out << height;
out << ',' //number seperator
out << width;
out << ',' //number seperator
out << name.size(); //serialize size of string
out << ',' //number seperator
out << name; //serialize characters of string
return out;
}
std::istream& MyClass::deserialize(std::istream &in) {
if (in) {
int len=0;
char comma;
in >> height;
in >> comma; //read in the seperator
in >> width;
in >> comma; //read in the seperator
in >> len; //deserialize size of string
in >> comma; //read in the seperator
if (in && len) {
std::vector<char> tmp(len);
in.read(tmp.data() , len); //deserialize characters of string
name.assign(tmp.data(), len);
}
}
return in;
}
You may also want to overload the stream operators for easier use.
std::ostream &operator<<(std::ostream& out, const MyClass &obj)
{obj.serialize(out); return out;}
std::istream &operator>>(std::istream& in, MyClass &obj)
{obj.deserialize(in); return in;}
Simply writing the binary contents of an object into a file is not only unportable but, as you've recognized, doesn't work for pointer data. You basically have two options: either you write a real serialization library, which handles std::strings properly by e.g. using c_str() to output the actual string to the file, or you use the excellent boost serialization library. If at all possible, I'd recommend the latter, you can then serialize with a simple code like this:
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/string.hpp>
class A {
private:
std::string s;
public:
template<class Archive>
void serialize(Archive& ar, const unsigned int version)
{
ar & s;
}
};
Here, the function serialize works for serializing and deserializing the data, depending on how you call it. See the documentation for more information.
The easiest serialization method for strings or other blobs with variable size is to serialize first the size as you serialize integers, then just copy the content to the output stream.
When reading you first read the size, then allocate the string and then fill it by reading the correct number of bytes from the stream.
An alternative is to use a delimiter and escaping, but requires more code and is slower both on serialization and deserialization (however the result can be kept human readable).
You'll have to use a more complicated method of serialization than casting a class to a char* and writing it to a file if your class contains any exogenous data (string does). And you're correct about why you're getting a segmentation fault.
I would make a member function that would take an fstream and read in the data from it as well as an inverse function which would take an fstream and write it's contents to it to be restored later, like this:
class MyClass {
pubic:
MyClass() : str() { }
void serialize(ostream& out) {
out << str;
}
void restore(istream& in) {
in >> str;
}
string& data() const { return str; }
private:
string str;
};
MyClass c;
c.serialize(output);
// later
c.restore(input);
You can also define operator<< and operator>> to work with istream and ostream to serialize and restore your class as well if you want that syntactic sugar.
Why not just something along the lines of:
std::ofstream ofs;
...
ofs << my_str;
and then:
std::ifstream ifs;
...
ifs >> my_str;
/*!
* reads binary data into the string.
* #status : OK.
*/
class UReadBinaryString
{
static std::string read(std::istream &is, uint32_t size)
{
std::string returnStr;
if(size > 0)
{
CWrapPtr<char> buff(new char[size]); // custom smart pointer
is.read(reinterpret_cast<char*>(buff.m_obj), size);
returnStr.assign(buff.m_obj, size);
}
return returnStr;
}
};
class objHeader
{
public:
std::string m_ID;
// serialize
std::ostream &operator << (std::ostream &os)
{
uint32_t size = (m_ID.length());
os.write(reinterpret_cast<char*>(&size), sizeof(uint32_t));
os.write(m_ID.c_str(), size);
return os;
}
// de-serialize
std::istream &operator >> (std::istream &is)
{
uint32_t size;
is.read(reinterpret_cast<char*>(&size), sizeof(uint32_t));
m_ID = UReadBinaryString::read(is, size);
return is;
}
};
I haven't coded C++ in a long time, but perhaps you could serialize an array of char.
Then, when you open your file, your string would just point to the array.
Just an idea.