I have a class called Game which contains the following:
vector<shared_ptr<A>> attr; // attributes
D diff; // differences
vector<shared_ptr<C>> change; // change
My question is, how can I write these (save) to a file and read/load it up later?
I thought about using a struct with these in it, and simply saving the struct but I have no idea where to start.
This is my attempt so far, with just trying to save change. I've read up a lot on the issue and my issue (well one of them, anyway) here seems to be that I am storing pointers which after closing the program would be invalid (compounded by the fact that I also free them before exiting).
/* Saves state to file */
void Game::saveGame(string toFile) {
ofstream ofs(toFile, ios::binary);
ofs.write((char *)&this->change, sizeof(C));
/* Free memory code here */
....
exit(0);
};
/* Loads game state from file */
void Game::loadGame(string fromFile) {
ifstream ifs(fromFile, ios::binary);
ifs.read((char *)&this->change, sizeof(C));
this->change.toString(); // display load results
};
Can anyone guide me in the right direction for serializing this data? I'd like to use only standard packages, so no boost.
Thanks.
I have no idea how is implemented classes A, C or D, but that is the first question: how to serialize an object of that class. For the C case, you need to implement something like this:
std::ostream& operator <<(std::ostream& os, const C& c) {
// ... code to serialize c to an output stream
return os;
}
std::istream& operator >>(std::istream& is, C& c) {
// ... code to populate c contents from the input stream
return is;
}
or, if you prefer, create a write() and read() function for that class.
Well, if you want to serialize a vector<shared_ptr<C>> looks obvious you don't want to serialize the pointer, but the contents. So you need to dereference each of those pointers and serialize. If the size of the vector is not known before loading it (i.e., is not always the same), you'll need to store that information. Then, you can create a pair of functions to serialize the complete vector:
std::ostream& operator <<(std::ostream& os, const std::vector<std::shared_ptr<C>>& vc) {
// serialize the size of the vector using << operator
// for each element of the vector, let it be called 'pc'
os << *pc << std::endl; // store the element pointed by the pointer, not the pointer.
return os;
}
std::istream& operator >>(std::istream& is, std::vector<std::shared_ptr<C>>& c) {
// read the size of the vector using >> operator
// set the size of the vector
// for each i < sizeo of the vector, let 'auto &pc = vc[i]' be a reference to the i-th element of the vector
C c; // temporary object
is >> c; // read the object stored in the stream
pc = std::make_shared<C>(c); // construct the shared pointer, assuming the class C has copy constructor
return is;
}
And then,
/* Saves state to file */
void Game::saveGame(string toFile) {
ofstream ofs(toFile);
ofs << change;
....
};
/* Loads game state from file */
void Game::loadGame(string fromFile) {
ifstream ifs(fromFile);
ifs >> change;
};
I know there are a lot of things you still need to resolve. I suggest you to investigate to resolve them so you understand well how to solve your problem.
Not only are you saving pointers, you're trying to save a shared_ptr but using the wrong size.
You need to write serialization functions for all your classes, taking care to never just write the raw bits of a non-POD type. It's safest to always implement member-by-member serialization for everything, because you never know what the future will bring.
Then handling collections of them is just a matter of also storing how many there are.
Example for the Cs:
void Game::save(ofstream& stream, const C& data)
{
// Save data as appropriate...
}
void Game::saveGame(string toFile) {
ofstream ofs(toFile, ios::binary);
ofs.write((char *)change.size(), sizeof(change.size());
for (vector<shared_ptr<C>>::const_iterator c = change.begin(); c != change.end(); ++c)
{
save(ofs, **c);
}
};
shared_ptr<C> Game::loadC(ofstream& stream)
{
shared_ptr<C> data(new C);
// load the object...
return data;
}
void Game::loadGame(string fromFile) {
change.clear();
size_t count = 0;
ifstream ifs(fromFile, ios::binary);
ifs.read((char *)&count, sizeof(count));
change.reserve(count);
for (int i = 0; i < count; ++i)
{
change.push_back(loadC(ifs));
}
};
All the error handling is missing of course - you would need to add that.
It's actually a good idea to at least start with text storage (using << and >>) instead of binary. It's easier to find bugs, or mess around with the saved state, when you can just edit it in a text editor.
Writing your own serialization is quite a challenge. Even if you do not use boost serializatoin I would recommend you learn how to use it and comprehend how it works rather than discovering it yourself.
When serializing you finally end up with a buffer of data of which content you have very vague idea. You have to save everything you need to be able to restore it. You read it chunk by chunk. Example (not compiled, not tested and not stylish ):
void save(ostream& out, const string& s)
{
out << s.size();
out.write(s.c_str(), s.size());
}
void load(istream& in, string& s)
{
unsigned len;
in >> len;
s.resize(len);
in.read((char*)s, len);
}
struct Game
{
void save(ostream& out)
{
player.save(out);
};
void load(istream& in)
{
player.load(in);
}
};
struct Player
{
void save(ostream& out)
{
// save in the same order as loading, serializing everything you need to read it back
save(out, name);
save(out, experience);
}
void load(istream& in)
{
load(in, name);
load(in, experience); //
}
};
I do not know why you would do it to yourself instead of using boost but those are some of the cases you should consider:
- type - you must figure out a way to know what "type of change" you actually have there.
- a string (vector, whatever) - size + data (then the first thing you read back from the string is the length, you resize it and copy the "length" number of characters)
- a pointer - save the data pointed by pointer, then upon deserialization you have to allocate it, construct it (usually default construct) and read back the data and reset the members to their respective values. Note: you have to avoid memory leakage.
- polymorphic pointer - ouch you have to know what type the pointer actually points to, you have to construct the derived type, save the values of the derived type... so you have to save type information
- null pointer... you have to distinguish null pointer so you know that you do not need to further read data from the stream.
- versioning - you have to be able to read a data after you added/removed a field
There is too much of it for you to get a complete answer.
Related
I am writing a program to that regularly stores and reads structs in the form below.
struct Node {
int leftChild = 0;
int rightChild = 0;
std::string value;
int count = 1;
int balanceFactor = 0;
};
How would I read and write nodes to a file? I would like to use the fstream class with seekg and seekp to do the serialization manually but I'm not sure how it works based off of the documentation and am struggling with finding decent examples.
[edit] specified that i do not want to use a serialization library.
This problem is known as serialization. Use a serializing library like e.g. Google's Protocol Buffers or Flatbuffers.
To serialize objects, you will need to stick to the concept that the object is writing its members to the stream and reading members from the stream. Also, member objects should write themselves to the stream (as well as read).
I implemented a scheme using three member functions, and a buffer:
void load_from_buffer(uint8_t * & buffer_pointer);
void store_to_buffer(uint8_t * & buffer_pointer) const;
unsigned int size_on_stream() const;
The size_on_stream would be called first in order to determine the buffer size for the object (or how much space it occupied in the buffer).
The load_from_buffer function loads the object's members from a buffer using the given pointer. The function also increments the pointer appropriately.
The store_to_buffer function stores the objects's members to a buffer using the given pointer. The function also increments the pointer appropriately.
This can be applied to POD types by using templates and template specializations.
These functions also allow you to pack the output into the buffer, and load from a packed format.
The reason for I/O to the buffer is so you can use the more efficient block stream methods, such as write and read.
Edit 1: Writing a node to a stream
The problem with writing or serializing a node (such a linked list or tree node) is that pointers don't translate to a file. There is no guarantee that the OS will place your program in the same memory location or give you the same area of memory each time.
You have two options: 1) Only store the data. 2) Convert the pointers to file offsets. Option 2) is very complicated as it may require repositioning the file pointer because file offsets may not be known ahead of time.
Also, be aware of variable length records like strings. You can't directly write a string object to a file. Unless you use a fixed string width, the string size will change. You will either need to prefix the string with the string length (preferred) or use some kind of terminating character, such as '\0'. The string length first is preferred because you don't have to search for the end of the string; you can use a block read to read in the text.
If you replace the std::string by a char buffer, you can use fwrite and fread to write/read your structure to and from disk as a fixed size block of information. Within a single program that should work ok.
The big bug-a-boo is the fact that compilers will insert padding between fields in order to keep the data aligned. That makes the code less portable as if a module is compiled with different alignment requirements the structure literally can be a different size, throwing your fixed size assumption out the door.
I would lean toward a well worn in serialization library of some sort.
Another approach would be to overload the operator<< and operator>> for the structure so that it knows how to save/load itself. That would reduce the problem to knowing where to read/write the node. In theory, your left and right child fields could be seek addresses to where the nodes actually reside, while a new field could hold the seek location of the current node.
When implementing your own serialization method, the first decision you'll have to make is whether you want the data on disk to be in binary format or textual format.
I find it easier to implement the ability to save to a binary format. The number of functions needed to implement that is small. You need to implement functions that can write the fundamental types, arrays of known size at compile time, dynamic arrays and strings. Everything else can be built on top of those.
Here's something very close to what I recently put into production code.
#include <cstring>
#include <fstream>
#include <cstddef>
#include <stdexcept>
// Class to write to a stream
struct Writer
{
std::ostream& out_;
Writer(std::ostream& out) : out_(out) {}
// Write the fundamental types
template <typename T>
void write(T number)
{
out_.write(reinterpret_cast<char const*>(&number), sizeof(number));
if (!out_ )
{
throw std::runtime_error("Unable to write a number");
}
}
// Write arrays whose size is known at compile time
template <typename T, uint64_t N>
void write(T (&array)[N])
{
for(uint64_t i = 0; i < N; ++i )
{
write(array[i]);
}
}
// Write dynamic arrays
template <typename T>
void write(T array[], uint64_t size)
{
write(size);
for(uint64_t i = 0; i < size; ++i )
{
write(array[i]);
}
}
// Write strings
void write(std::string const& str)
{
write(str.c_str(), str.size());
}
void write(char const* str)
{
write(str, std::strlen(str));
}
};
// Class to read from a stream
struct Reader
{
std::ifstream& in_;
Reader(std::ifstream& in) : in_(in) {}
template <typename T>
void read(T& number)
{
in_.read(reinterpret_cast<char*>(&number), sizeof(number));
if (!in_ )
{
throw std::runtime_error("Unable to read a number.");
}
}
template <typename T, uint64_t N>
void read(T (&array)[N])
{
for(uint64_t i = 0; i < N; ++i )
{
read(array[i]);
}
}
template <typename T>
void read(T*& array)
{
uint64_t size;
read(size);
array = new T[size];
for(uint64_t i = 0; i < size; ++i )
{
read(array[i]);
}
}
void read(std::string& str)
{
char* s;
read(s);
str = s;
delete [] s;
}
};
// Test the code.
#include <iostream>
void writeData(std::string const& file)
{
std::ofstream out(file);
Writer w(out);
w.write(10);
w.write(20.f);
w.write(200.456);
w.write("Test String");
}
void readData(std::string const& file)
{
std::ifstream in(file);
Reader r(in);
int i;
r.read(i);
std::cout << "i: " << i << std::endl;
float f;
r.read(f);
std::cout << "f: " << f << std::endl;
double d;
r.read(d);
std::cout << "d: " << d << std::endl;
std::string s;
r.read(s);
std::cout << "s: " << s << std::endl;
}
void testWriteAndRead(std::string const& file)
{
writeData(file);
readData(file);
}
int main()
{
testWriteAndRead("test.bin");
return 0;
}
Output:
i: 10
f: 20
d: 200.456
s: Test String
The ability to write and read a Node is very easily implemented.
void write(Writer& w, Node const& n)
{
w.write(n.leftChild);
w.write(n.rightChild);
w.write(n.value);
w.write(n.count);
w.write(n.balanceFactor);
}
void read(Reader& r, Node& n)
{
r.read(n.leftChild);
r.read(n.rightChild);
r.read(n.value);
r.read(n.count);
r.read(n.balanceFactor);
}
The process you are referring to are known as serialization. I'd recommend Cereal at http://uscilab.github.io/cereal/
It supports both json, xml and binary serialization and is very easy to use (with good examples).
(Unfortunately it does not support my favourite format yaml)
I'm writing data (structure) into file using vector, and when I attempt to retrieve data using vector iterator and it gives me: "Vector iterator is not dereferenceable."
This is my code:
void CProgram_1_STLDlg::OnBnClickedBtnView()
{
// TODO: Add your control notification handler code here
CFile file;
CFileException e;
studentVector::iterator sit;
studentVector::iterator sBegin = sVector.begin();
studentVector::iterator sEnd = sVector.end();
CString path = _T("D:\\Student.txt");
if ( file.Open(path, CFile::modeRead, &e) ) {
while ( file.Read( (char *)&sVector, sizeof(sVector)) ) {
AfxMessageBox(_T("File opened in read mode."), MB_ICONINFORMATION);
AfxMessageBox(_T("ID:\t")+sit->id+L"\nName:\t"
+sit->name+L"\nMarks:\t"+sit->marks+L
"\nPercentage:\t"+sit->per+L"\nState:\t"+sit->state);
sit++;
}
//file.Read( (char *)&sData, sizeof(sData));
/*for ( sIterator = sVector.begin(); sIterator != sVector.end(); sIterator++ ) {
//AfxMessageBox(_T("ID:\t")+sIterator->id+L
"\nName:\t"+sIterator->name+L"\nMarks:\t"
+sIterator->marks+L"\nPercentage:\t"+sIterator->per+L
"\nState:\t"+sIterator->state);
//AfxMessageBox(_T("Hello..Testing...!"));
}
*/
} else {
AfxMessageBox(_T("Error! Unable to open file."), MB_ICONERROR);
}
}
Now I don't know how to resolve this error.
Note: Some of links I refer which google gave me, but I couldn't able to solve my problem.
You cannot simply overwrite the memory of a vector. That is pretty much guaranteed to corrupt your process.
Furthermore, you never assign anything to sit and yet expect it to contain something sensible.
You need to parse the data in Student.txt and use vector's member functions to fill it with sensible data. The assignment will probably tell you what the file looks like so that you can parse it.
A simple vector like
vector<char> cvec
could be overwritten
so something like
vector<char> cvec;
cvec.resize(100);
for(char i=0;i<100;i++)
cvec[i]=i;
will work.
If you resize to correct size. Otherwise you will corrupt memory
sizeof(sVector) will deliver the size of the vector class.
this is not related to the data since data inside the vector class is nothing more than a pointer.
example:
class simpleVector;
{
public:
simpleVector(unigned int size)
{
p=new int[size];
}
int* p;
}
func()
{
simpleVector v1(10);
simpleVector v2(100000);
printf("size v1= %d, Size v2= %d", sizeof(v1),sizeog(v2));
}
I have not checked, what sizeof will deliver for this class, but it definitely will be constant. Independent from the size that is given to constructor
An Iterator is an accessor to the Vector
but it needs to be initialized.
In the code above sit is not assigned to something. So you are not able to access something valid.
from the code line
AfxMessageBox(_T("ID:\t")+sit->id+L"\nName:\t"+sit->name+L"\nMarks:\t"+sit->marks+L"\nPercentage:\t"+sit->per+L"\nState:\t"+sit->state);
I see the vector shall contain a complex data type build from several strings.
so a vector element probably looks like
class student
{
std::string id;
std::string name;
std::string marks;
std::string per;
std::string state;
};
this is in minimum the information hold by each vector element.
usually strings have the property to have different length.
While id might be always of same length name probably don't.
Since it is not fixed length
even
file.Read( (char *)&sVector, sizeof(student))
would not work.
so I would suggest to add a reader to the 'Student' Class:
class student
{
std::string id;
std::string name;
std::string marks;
std::string per;
std::string state;
bool ReadElemFromFile(CFile& file)
{
id=ReadStringFromFile(file);
name=ReadStringFromFile(file);
marks=ReadStringFromFile(file);
per=ReadStringFromFile(file);
state=ReadStringFromFile(file);
if(id.empty()||name.empty()||marks.empty()||per.empty()||state.empty())
return false;
return true;
}
std::string ReadStringFromFile(CFile% file)
{
char c;
std::string s;
do
{
file.read(&c,1);
s+=c;
}
while(c!='\0')
return s;
}
};
I know reading that way is not the most performant way to do it, but it shows,that the string terminator stored to file indicates the length of each string
now back to your code
void CProgram_1_STLDlg::OnBnClickedBtnView()
{
// TODO: Add your control notification handler code here
CFile file;
CFileException e;
student* sit=new Student;
studentVector.clear();
CString path = _T("D:\\Student.txt");
if ( file.Open(path, CFile::modeRead, &e) ) {
while ( sit->ReadElemFromFile(CFile& file)) {
AfxMessageBox(_T("File opened in read mode."), MB_ICONINFORMATION);
AfxMessageBox(_T("ID:\t")+sit->id+L"\nName:\t"+sit->name+L"\nMarks:\t"+sit->marks+L"\nPercentage:\t"+sit->per+L"\nState:\t"+sit->state);
studentVector.push_back(*sit);
}
} else {
AfxMessageBox(_T("Error! Unable to open file."), MB_ICONERROR);
}
delete stud;
}
..."attempt to retrieve data using vector iterator and it gives me Vector iterator is not dereferenceable"...
Iterators are pointer-like objects, however unlike raw pointers, they prevent dereferencing (accessing of the value they point to) if they are "dangling".
In your case iterator sit is not initialized, not as, for example iterator sBegin = sVector.begin();, that is assigned to point to the beginning of the vector sVector.
Thus when you try to access an iterator that does not point to a valid value, you get an error.
In addition to that, to store an element to a vector you should use its member functions, not passing its address, as you do in your while loop.
I'm currently learning C++ (Coming from Java) and I'm trying to understand how to use IO streams properly in C++.
Let's say I have an Image class which contains the pixels of an image and I overloaded the extraction operator to read the image from a stream:
istream& operator>>(istream& stream, Image& image)
{
// Read the image data from the stream into the image
return stream;
}
So now I'm able to read an image like this:
Image image;
ifstream file("somepic.img");
file >> image;
But now I want to use the same extraction operator to read the image data from a custom stream. Let's say I have a file which contains the image in compressed form. So instead of using ifstream I might want to implement my own input stream. At least that's how I would do it in Java. In Java I would write a custom class extending the InputStream class and implementing the int read() method. So that's pretty easy. And usage would look like this:
InputStream stream = new CompressedInputStream(new FileInputStream("somepic.imgz"));
image.read(stream);
So using the same pattern maybe I want to do this in C++:
Image image;
ifstream file("somepic.imgz");
compressed_stream stream(file);
stream >> image;
But maybe that's the wrong way, don't know. Extending the istream class looks pretty complicated and after some searching I found some hints about extending streambuf instead. But this example looks terribly complicated for such a simple task.
So what's the best way to implement custom input/output streams (or streambufs?) in C++?
Solution
Some people suggested not using iostreams at all and to use iterators, boost or a custom IO interface instead. These may be valid alternatives but my question was about iostreams. The accepted answer resulted in the example code below. For easier reading there is no header/code separation and the whole std namespace is imported (I know that this is a bad thing in real code).
This example is about reading and writing vertical-xor-encoded images. The format is pretty easy. Each byte represents two pixels (4 bits per pixel). Each line is xor'd with the previous line. This kind of encoding prepares the image for compression (usually results in lot of 0-bytes which are easier to compress).
#include <cstring>
#include <fstream>
using namespace std;
/*** vxor_streambuf class ******************************************/
class vxor_streambuf: public streambuf
{
public:
vxor_streambuf(streambuf *buffer, const int width) :
buffer(buffer),
size(width / 2)
{
previous_line = new char[size];
memset(previous_line, 0, size);
current_line = new char[size];
setg(0, 0, 0);
setp(current_line, current_line + size);
}
virtual ~vxor_streambuf()
{
sync();
delete[] previous_line;
delete[] current_line;
}
virtual streambuf::int_type underflow()
{
// Read line from original buffer
streamsize read = buffer->sgetn(current_line, size);
if (!read) return traits_type::eof();
// Do vertical XOR decoding
for (int i = 0; i < size; i += 1)
{
current_line[i] ^= previous_line[i];
previous_line[i] = current_line[i];
}
setg(current_line, current_line, current_line + read);
return traits_type::to_int_type(*gptr());
}
virtual streambuf::int_type overflow(streambuf::int_type value)
{
int write = pptr() - pbase();
if (write)
{
// Do vertical XOR encoding
for (int i = 0; i < size; i += 1)
{
char tmp = current_line[i];
current_line[i] ^= previous_line[i];
previous_line[i] = tmp;
}
// Write line to original buffer
streamsize written = buffer->sputn(current_line, write);
if (written != write) return traits_type::eof();
}
setp(current_line, current_line + size);
if (!traits_type::eq_int_type(value, traits_type::eof())) sputc(value);
return traits_type::not_eof(value);
};
virtual int sync()
{
streambuf::int_type result = this->overflow(traits_type::eof());
buffer->pubsync();
return traits_type::eq_int_type(result, traits_type::eof()) ? -1 : 0;
}
private:
streambuf *buffer;
int size;
char *previous_line;
char *current_line;
};
/*** vxor_istream class ********************************************/
class vxor_istream: public istream
{
public:
vxor_istream(istream &stream, const int width) :
istream(new vxor_streambuf(stream.rdbuf(), width)) {}
virtual ~vxor_istream()
{
delete rdbuf();
}
};
/*** vxor_ostream class ********************************************/
class vxor_ostream: public ostream
{
public:
vxor_ostream(ostream &stream, const int width) :
ostream(new vxor_streambuf(stream.rdbuf(), width)) {}
virtual ~vxor_ostream()
{
delete rdbuf();
}
};
/*** Test main method **********************************************/
int main()
{
// Read data
ifstream infile("test.img");
vxor_istream in(infile, 288);
char data[144 * 128];
in.read(data, 144 * 128);
infile.close();
// Write data
ofstream outfile("test2.img");
vxor_ostream out(outfile, 288);
out.write(data, 144 * 128);
out.flush();
outfile.close();
return 0;
}
The proper way to create a new stream in C++ is to derive from std::streambuf and to override the underflow() operation for reading and the overflow() and sync() operations for writing. For your purpose you'd create a filtering stream buffer which takes another stream buffer (and possibly a stream from which the stream buffer can be extracted using rdbuf()) as argument and implements its own operations in terms of this stream buffer.
The basic outline of a stream buffer would be something like this:
class compressbuf
: public std::streambuf {
std::streambuf* sbuf_;
char* buffer_;
// context for the compression
public:
compressbuf(std::streambuf* sbuf)
: sbuf_(sbuf), buffer_(new char[1024]) {
// initialize compression context
}
~compressbuf() { delete[] this->buffer_; }
int underflow() {
if (this->gptr() == this->egptr()) {
// decompress data into buffer_, obtaining its own input from
// this->sbuf_; if necessary resize buffer
// the next statement assumes "size" characters were produced (if
// no more characters are available, size == 0.
this->setg(this->buffer_, this->buffer_, this->buffer_ + size);
}
return this->gptr() == this->egptr()
? std::char_traits<char>::eof()
: std::char_traits<char>::to_int_type(*this->gptr());
}
};
How underflow() looks exactly depends on the compression library being used. Most libraries I have used keep an internal buffer which needs to be filled and which retains the bytes which are not yet consumed. Typically, it is fairly easy to hook the decompression into underflow().
Once the stream buffer is created, you can just initialize an std::istream object with the stream buffer:
std::ifstream fin("some.file");
compressbuf sbuf(fin.rdbuf());
std::istream in(&sbuf);
If you are going to use the stream buffer frequently, you might want to encapsulate the object construction into a class, e.g., icompressstream. Doing so is a bit tricky because the base class std::ios is a virtual base and is the actual location where the stream buffer is stored. To construct the stream buffer before passing a pointer to a std::ios thus requires jumping through a few hoops: It requires the use of a virtual base class. Here is how this could look roughly:
struct compressstream_base {
compressbuf sbuf_;
compressstream_base(std::streambuf* sbuf): sbuf_(sbuf) {}
};
class icompressstream
: virtual compressstream_base
, public std::istream {
public:
icompressstream(std::streambuf* sbuf)
: compressstream_base(sbuf)
, std::ios(&this->sbuf_)
, std::istream(&this->sbuf_) {
}
};
(I just typed this code without a simple way to test that it is reasonably correct; please expect typos but the overall approach should work as described)
boost (which you should have already if you're serious about C++), has a whole library dedicated to extending and customizing IO streams: boost.iostreams
In particular, it already has decompressing streams for a few popular formats (bzip2, gzlib, and zlib)
As you saw, extending streambuf may be an involving job, but the library makes it fairly easy to write your own filtering streambuf if you need one.
Don't, unless you want to die a terrible death of hideous design. IOstreams are the worst component of the Standard library - even worse than locales. The iterator model is much more useful, and you can convert from stream to iterator with istream_iterator.
I agree with #DeadMG and wouldn't recommend using iostreams. Apart from poor design the performance is often worse than that of plain old C-style I/O. I wouldn't stick to a particular I/O library though, instead, I'd create an interface (abstract class) that has all required operations, for example:
class Input {
public:
virtual void read(char *buffer, size_t size) = 0;
// ...
};
Then you can implement this interface for C I/O, iostreams, mmap or whatever.
It is probably possible to do this, but I feel that it's not the "right" usage of this feature in C++. The iostream >> and << operators are meant for fairly simple operations, such as wriitng the "name, street, town, postal code" of a class Person, not for parsing and loading images. That's much better done using the stream::read() - using Image(astream);, and you may implement a stream for compression, as descrtibed by Dietmar.
I'm trying to keep objects including vectors of objects in a binary file.
Here's a bit of the load from file code:
template <class T> void read(T* obj,std::ifstream * file) {
file->read((char*)(obj),sizeof(*obj));
file->seekg(int(file->tellg())+sizeof(*obj));
}
void read_db(DB* obj,std::ifstream * file) {
read<DB>(obj,file);
for(int index = 0;index < obj->Arrays.size();index++) {
std::cin.get(); //debugging
obj->Arrays[0].Name = "hi"; //debugging
std::cin.get(); //debugging
std::cout << obj->Arrays[0].Name;
read<DB_ARRAY>(&obj->Arrays[index],file);
for(int row_index = 0;row_index < obj->Arrays[index].Rows.size();row_index++) {
read<DB_ROW>(&obj->Arrays[index].Rows[row_index],file);
for(int int_index = 0;int_index < obj->Arrays[index].Rows[row_index].i_Values.size();int_index++) {
read<DB_VALUE<int>>(&obj->Arrays[index].Rows[row_index].i_Values[int_index],file);
}
}
}
}
And here's the DB/DB_ARRAY classes
class DB {
public:
std::string Name;
std::vector<DB_ARRAY> Arrays;
DB_ARRAY * operator[](std::string);
DB_ARRAY * Create(std::string);
};
class DB_ARRAY {
public:
DB* Parent;
std::string Name;
std::vector<DB_ROW> Rows;
DB_ROW * operator[](int);
DB_ROW * Create();
DB_ARRAY(DB*,std::string);
DB_ARRAY();
};
So now the first argument to the read_db function would have correct values, and the vector Arrays on the object has the correct size, However if I index any value of any object from obj->Arrays it's going to throw the access violation exception.
std::cout << obj->Arrays[0].Name; // error
std::cout << &obj->Arrays[0]; // no error
The later always prints the same address, so when I save an object casted to char* does it save the address of it too?
As various commenters pointed out, you cannot simply serialize a (non-POD) object by saving / restoring it's memory.
The usual way to implement serialization is to implement a serialization interface on the classes. Something like this:
struct ISerializable {
virtual std::ostream& save(std::ostream& os) const = 0;
virtual std::istream& load(std::istream& is) = 0;
};
You then implement this interface in your serializable classes, recursively calling save and load on any members referencing other serializable classes, and writing out any POD members. E.g.:
class DB_ARRAY : public ISerializable {
public:
DB* Parent;
std::string Name;
std::vector<DB_ROW> Rows;
DB_ROW * operator[](int);
DB_ROW * Create();
DB_ARRAY(DB*,std::string);
DB_ARRAY();
virtual std::ostream& save(std::ostream& os) const
{
// serialize out members
return os;
}
virtual std::istream& load(std::istream& is)
{
// unserialize members
return os;
}
};
As count0 pointed out, boost::serialization is also a great starting point.
What is the format of the binary data in the file? Until you specify
that, we can't tell you how to write it. Basically, you have to specify
a format for all of your data types (except char), then write the code
to write out that format, byte by byte (or generate it into a buffer);
and on the other side, to read it in byte by byte, and reconstruct it.
The C++ standard says nothing (or very little) about the size and
representation of the data types, except that sizeof(char) must be
1, and that unsigned char must be a pure binary representation over
all of the bits. And on the machines I have access today (Sun Sparc and
PC's), only the character types have a common representation. As for
the more complex types, the memory used in the value representation
might not even be contiguous: the bitwise representation of an
std::vector, for example, is usually three pointers, with the actual
values in the vector being found somewhere else entirely.
The functions istream::read and ostream::write are
designed for reading data into a buffer for manual parsing, and writing
a pre-formatted buffer. The fact that you need to use a
reinterpret_cast to use them otherwise should be a good indication
that it won't work.
I know I can use:
MyGame game; // the game object
//
ofstream out("mygame.bin", ios::binary);
out.write((char *)&game, sizeof(MyGame));
to save and load the game, but what if I have pointers inside MyGame structure? will the pointers just be saved but not the data it points to?
and: how to solve this?
You can't just write pointers to a stream and expect it to be magically done. You need to implement save/load methods in your objects. E.g:
class Serializable
{
virtual void save(std::ofstream& _out) const = 0;
virtual void load(std::ifstream& _in) = 0;
}; // eo class Serializable
// some game object
class MyObject : public Serializable
{
int myInt;
std::string myString;
virtual void save(std::ofstream& _out) const
{
_out << myInt << myString;
}; // eo save
virtual void load(std::ifstream& _in)
{
_in >> myInt >> myString;
}; // eo load
}; // eo class SomeObject
class MyGame : public Serializable
{
MyObject a;
MyObject b;
virtual void save(std::ofstream& _out) const
{
a.save(_out);
b.save(_out);
}; // eo save
virtual void load(std::ifstream& _in)
{
a.load(_in);
b.load(_in);
}; // eo load
}; // eo class MyGame
Assuming you have not overridden char * cast, yes this will most probably save only pointer and not data.
What you need is Serialization of your object. You can provide a method to marshal the state of object in a bit stream and write that out. And you also need to have method to restore the state back.
You may read more about serialization on wikipedia
Boost has a serialization library, with built in support for deep pointer save and restore, and proper serialization of pointers to shared data.
It's a rather extensive library, but you don't need to write that much code to start using it in your own projects. Well worth the learning effort for anything but the simplest serialization requirements in my opinion.
You could overload the stream out operator (<<) and stream out each individual field (and vice versa)
EDIT: here is a complete example...
#include <iostream>
#include <fstream>
#include <map>
using namespace std;
template <typename T>
void serialize(ostream& str, const T& field)
{
str.rdbuf()->sputn(reinterpret_cast<const char*>(&field), sizeof(T));
}
template <typename T>
void deserialize(istream& str, T& field)
{
str.rdbuf()->sgetn(reinterpret_cast<char*>(&field), sizeof(T));
}
class MyGame
{
public:
MyGame() : a(), b() {}
MyGame(int av, int bv) : a(av), b(bv) {}
friend ostream& operator<<(ostream& str, MyGame const& game);
friend istream& operator>>(istream& str, MyGame& game);
int getA() const { return a; }
int getB() const { return b; }
private:
int a;
int b;
};
ostream& operator<<(ostream& str, MyGame const& game)
{
serialize(str, game.a);
serialize(str, game.b);
return str;
}
istream& operator>>(istream& str, MyGame& game)
{
deserialize(str, game.a);
deserialize(str, game.b);
return str;
}
int main(void)
{
{
ofstream fout("test.bin", ios::binary);
MyGame game(10, 11);
fout << game;
}
{
ifstream fin("test.bin", ios::binary);
MyGame game;
fin >> game;
cout << "game.a: " << game.getA() << ", game.b: " << game.getB() << endl;
}
return 0;
}
You must understand the issues with this approach though, such as the resulting file will be platform specific (i.e. non-portable) etc.
Try game.serialize(out);. In your serialize member function call serialize of your pointer members.
Make a serializing function per type that needs to be persistent.
Call this for each member.
It is actually similar to serializing over network or visualizing for debug-purposes.
boost.serialize can help you.
"Naive" serialization that just dumps the value of pointers is never going to work because, when deserializing, those pointers will be invalid.
The general approach to this kind of problem would go like this:
Have each object that can be serialized in your game implement a serialize() virtual function (I am assuming all such objects will ultimately derive from the same base class).
Have the base class implement a get_serialized_id() public function. This function will use a static auto-incremented variable to generate a unique id for each object instance, but only the first time it is called (subsequent calls will just return the existing value).
Now, when serializing:
Start with a std::map<int, YourBaseClass*>. Add your game object to this map, using the value returned by get_serialized_id() for the key.
While the map contains objects that have not been serialized yet:
Take the first such object.
Serialize its get_serialized_id().
Serialize it by calling its implementation for serialize(). Have it persist its primitive data as usual. For data available through pointers, call get_serialized_id() on each object pointed to and just serialize the number returned from it. Also, add that object to the map.
This will result in a bunch of objects being serialized (in a "random" order) along with each one's "random" id.
When deserializing:
Start with a std::map<int, YourBaseClass*>. Read the first item in your saved file.
For each object pointed to by this first object, you know a unique id (this is what you serialized instead of a pointer). Fetch the item with this id from the saved file and deserialize it.
Recursively do this until all items have been fetched and deserialized from the saved file.
As each item has all its dependences deserialized in step 3 above, instantiate it as an object and add it to the map.
This will enable you to grab a pointer from the map given an item's id, which you can now use to set the pointer members of objects dependent on this item.
When the recursion ends, the last object in the map will be your main "game" object with all its pointers ready to go.
What you did is shallow copy, if you have pointers in your MyGame class, then a deep copy is a MUST!.
I suggest implementing a function or a set of functiions inside MyGame that will take care of saving its own data to a file,and you will only need to call it.
Thanks everyone for the fast and good answers, but a buddy of mine (who is helping me on this) told me we should do it in another way.
I just save the basics of the object and we recreate the rest in a function.
It's a card-game and to save the stack of cards we'll be saving the ID of the card only (not the objects) and just re-initializing each card when we read in the ID from the file.