I have my struct:
struct a
{
int x;
float f;
double d;
char c;
char s[50];
};
and I wish append each time into my timer schedule into a binary file.
// declaration
std::ofstream outFile;
// constructor:
outFile.open( "save.dat", ios::app );
// tick:
outFile << a << endl;
but inside the save.dat appears only this:
0C3A0000..0C3A0000..0C3A0000..0C3A0000..0C3A0000..0C3A0000..0C3A0000..0C3A0000..0C3A0000..
thanks in advance
What you're currently doing is writing the address of the struct definition.
What you want to do is use ostream::write
outfile.write(reinterpret_cast<char*>(&myStruct), sizeof(a));
This will work as long as your struct is a POD (Plain Old Data) type (which your example is). POD type means that all members are of fixed size.
If you on the other hand have variable sized members then you would need to write out each member one by one.
A sensible way to serialize custom objects is to overload your own output stream operator:
std::ostream & operator<<(std::ostream & o, const a & x)
{
o.write(reinterpret_cast<char*>(&x.x), sizeof(int));
o.write(reinterpret_cast<char*>(&x.f), sizeof(float));
/* ... */
return o;
}
a x;
std::ofstream ofile("myfile.bin", std::ios::binary | std::ios::app);
ofile << a;
This is still platform-dependent, so to be a bit safer, you should probably use fixed-width data types like int32_t etc.
It might also not be the best idea semantically to use << for binary output, since it's often used for formatted output. Perhaps a slightly safer method would be to write a function void serialize(const a &, std::ostream &);
Related
I noticed, that when using:
std::vector<int> v(100000);
...
std::ofstream outfile("outfile.dat", std::ios::out | std::ofstream::binary);
std::copy(v.begin(), v.end(), std::ostream_iterator<int>(outfile));
outfile.close();
my std::vector<int> is not serialized as raw bytes data (4 bytes per int) but as string, i.e. the string representation of each integer is saved to disk, which I don't want.
How to save a std::vector<int> as binary data?
(Note: I'd like to learn it with standard C++03, before learning new methods for it).
To write binary data, use std::ostream::write() instead of std::ostream_iterator (which uses operator<< internally, thus formatted output), eg:
std::vector<int> v(100000);
...
std::ofstream outfile("outfile.dat", std::ofstream::binary);
outfile.write(reinterpret_cast<const char*>(v.data() /* or &v[0] pre-C++11 */), sizeof(int) * v.size());
outfile.close();
std::ostream_iterator writes values to the stream using its operator<<. Elements are written as if you used outfile << value for each member of the vector, which means converting values to text.
So, what you want to do instead is define a class that serializes itself to the stream in a binary representation, eg:
std::copy(v.begin(), v.end(), std::ostream_iterator<BinaryInt>(outfile));
^^^^^^^^^
Now you have to define the BinaryInt type so that it can be constructed by an int value but serialize itself via operator<< appropriately:
struct BinaryInt
{
int value;
BinaryValue(int v): value(v) {}
friend std::ostream& operator<<(std::ostream& str, BinaryInt const& bi)
{
// convert bi.value into a binary representation.
// Note C++ does not define a specific size for int.
// Nor does it define an endianess.
// Nor does it define a specific representation.
// So to be cross platform/OS/compiler you will need to define these
// and convert the integer into this representation.
//
// return str.write(<data>, <size>);
//
// If this is just a test the following would work
// but is extremely brittle for the long term.
return str.write(reinterpret_cast<const char*>(&bi.value), sizeof(bi.value));
}
};
Might I recommend a saner way of doing it by using Protobufs? I will not type the code, but if you are working on a project, do not reinvent the wheel.
Using a protobuf would allow you to save the "type" of your data along with the data, and it would help you to extend your code with minimal fuss.
In a c++ textbook I learn that I should be able to do this (write/read a user-defined type to a file, in binary mode):
struct mystruct
{
int a;
double b;
};
void main()
{
struct mystruct s = {1,2.0};
std::ofstream out("binary.dat", std::ios::out | std::ios::binary);
out.write(reinterpret_cast< const char* > (&s), sizeof(struct mystruct));
}
The code compiles and runs but the resultant file is cannot be read by text-editors. then I use binary mode to read the file:
std::ifstream in("binary.dat", std::ios::in | std::ios::binary);
struct mystruct t;
in.read( reinterpret_cast<char*> (&t), sizeof(struct mystruct));
However it turns out it's unsuccessful (it doesn't get the numbers 1 and 2.0). Maybe I interpret the textbook wrongly..., or maybe something wrong with the code?
but the resultant file is cannot be read by text-editors
That's because you didn't write text to the file. The data cannot be interpreted meaningfully in any character encoding. You can use hexdump or similar tool for a meaningful representation.
Maybe I interpret the textbook wrongly..., or maybe something wrong with the code?
I don't think so. There's nothing wrong with the pieces of code individually except weird formatting and redundant keywords. Besides non-portability of the file of course. And the invalid return type of main.
However, perhaps what you did was a single program like this:
// ...
std::ofstream out("binary.dat", std::ios::binary);
out.write(reinterpret_cast<const char*>(&s), sizeof(s));
std::ifstream in("binary.dat", std::ios::binary);
in.read(reinterpret_cast<char*>(&t), sizeof(t));
// ...
There's a bug here. As you can see from the documentation, ostream::write writes to it's associated stream buffer. It's not until ostream::flush is called that the buffer is written to output. So, in the above code, the file is empty at the time it is read. This can obviously be fixed by calling out.flush() before reading the file.
As mentioned in comments, this is not at all portable. What that means is that a file written by your program on one computer may not be readable by your program on another computer. Even on the same computer, two programs compiled with different compiler options may not have the same memory layout for the class.
Why not do the standard way. It's simple and easy to understand
struct mystruct {
int a;
double b;
friend std::ostream & operator <<(std::ostream & out, mystruct & mys) {
out << mys.a << " " << mys.b << std::endl;
return out;
}
friend std::istream & operator >>(std::istream & in, mystruct & mys) {
in >> mys.a >> mys.b;
return in;
}
};
The usage will be simple now. For example
mystruct s;
To read
std::ifstream in("binary.dat");
in >> s;
To write
std::ofstream out("binary.dat");
out << s;
You can also overload std::string operator inside the struct if you want to serialize
operator std::string() const {
std::stringstream ss;
ss << a << " " << b;
return ss.str();
}
I am new to c++ and specifically file handling.
I made this code.
#include <iostream>
#include <fstream>
using namespace std;
class student
{
public:
char s;
int age;
};
int main (void)
{
student a;
a.s = 'a';
a.age = 1;
ofstream myfile;
myfile.open("a1.txt",ios::app);
myfile.write(reinterpret_cast<char*>(&a),sizeof(student));
return 0;
}
When I opened the file a1.txt, it had the character a correctly in there, but for the integer it had some weird encoding and a message that if you continue to edit this file, it will be corrupted. Can't I write an object to a file containing an integer and a character or a string as well?
You're writing the binary representation of student into the file. The character will come out as expected; but the int will be the bytes used to represent the value, not a readable number.
If you want the output to be formatted as readable text, use formatted output:
myfile << a.s << ' ' << a.age << '\n';
For convenience, you could overload the operator for your class:
ostream & operator<<(ostream & os, student const & a) {
return os << a.s << ' ' << a.age;
}
myfile << a << '\n';
For more complex structures, you might consider the Boost.Serialization library. Or you might do what I tend to do, with tuples instead of plain structures, and variadic templates to read and write them, but that might be rather more fiddly than you'd like.
I would define your own << operator that handles your custom type.
#include <iostream>
#include <fstream>
using namespace std;
class student
{
char s;
int age;
};
ostream& operator<<(ostream &output, const student &o)
{
output << o.s << " " << o.age;
return output;
}
int main (void)
{
student a;
a.s = 'a';
a.age = 1;
ofstream myfile;
myfile.open("a1.txt",ofstream::app);
ofstream << a;
return 0;
}
When you called myfile.write(reinterpret_cast<char*>(&a),sizeof(student)); it was not converting your struct to a human-readable string then writing it to file. In reality, it was interpreting the memory of your struct as a series of characters then writing it to file.
You actually did write the int to the file, but not readable (123 => '1' '2' '3')
but the 4 (or 8) byte of that int. Your program can read the value back too,
the only probem is that we humans can´t read that form well.
Concering Strings:Just writing the whole struct will probably fail
(depending on the exact type of the string variable etc.), because the
"string" often stores only a pointer in the struct (which points to some
memory elsewhere, and this other memory isn´t written automatically to the file)
To be safe, write each variable of the struct explicitely (and handle different
var type appropiately) instead of writing them all together.
This way, things like different variable ordering and struct padding
can´t cause problems too. Other pitfalls to remember are different int
sizes and endianess on different computers... serialization isn´t trivial.
I have some data in a buffer pointed to by a const char* pointer. The data is just an ASCII string. I know its size. I would like to be able to read it in the same way data is read from streams. I'm looking for a solution that would allow me to write code like this:
// for example, data points to a string "42 3.14 blah"
MemoryStreamWrapper in(data, data_size);
int x;
float y;
std::string w;
in >> x >> y >> w;
Important condition: the data must not be copied or altered in any way (otherwise I'd just use a string stream. To my best knowledge, it isn't possible to create a string stream from a const char pointer without copying the data.)
The way to do this is to create a suitable stream buffer. This can, e.g., be done like this:
#include <streambuf>
#include <istream>
struct membuf: std::streambuf {
membuf(char const* base, size_t size) {
char* p(const_cast<char*>(base));
this->setg(p, p, p + size);
}
};
struct imemstream: virtual membuf, std::istream {
imemstream(char const* base, size_t size)
: membuf(base, size)
, std::istream(static_cast<std::streambuf*>(this)) {
}
};
The only somewhat awkward thing is the const_cast<char*>() in the stream buffer: the stream buffer won't change the data but the interface still requires char* to be used, mainly to make it easier to change the buffer in "normal" stream buffers. With this, you can use imemstream as a normal input stream:
imemstream in(data, size);
in >> value;
The only way would be to subclass std::istream (which also requires subclassing std::streambuf) to create your own stream class that reads from constant memory.
It's not as easy as it sounds because the the C++ standard library stream classes are pretty messy and badly designed. I don't think it's worth it unless you need it to scale a lot.
I have data in stl containers (vector). Each node in the vector is a structure which also contains stl strings.
struct record
{
string name;
string location;
int salary;
}
vector< record > employees;
I want to serialize employees but I also want to encrypt it before serializing.
my encryption function looks like this:
Encode(const char * inBfr, const int in_size, char ** outBfr, int& out_size )
By searching it looks like the stl standard doesn't require the memory of my structure to be contiguous so I can't just grab the memory of employees variable. Is there any other smart way that I can use this encoding function with my stl based structures/container? It is good for me that Encode function works in plain char * buffers so I know exactly what goes in and out but stl structures are not and I am tring to find a nice way so I can use stl with this function.
I am also opening to using any other stl containers if that helps.
Although the element in the std::vector<T> are guaranteed to be laid out contiguously, this doesn't really help: the record you have may include padding and, more importantly, will store the std::string's content external to the std::string object (in case the small string optimization is used, the value may be embedded inside the std::string but it will also contain a couple of bytes which are not part of the std::strings value). Thus, you best option is to format your record and encrypt the formatted string.
The formatting is straight forward but personally I would encapsulate the encoding function into a simple std::streambuf so that the encryption can be done by a filtering stream buffer. Given the signature you gave, this could look something like this:
class encryptbuf
: public std::streambuf {
std::streambuf* d_sbuf;
char d_buffer[1024];
public:
encryptbuf(std::streambuf* sbuf)
: d_sbuf(sbuf) {
this->setp(this->d_buffer, this->d_buffer + sizeof(this->d_buffer) - 1);
}
int overflow(int c) {
if (c != std::char_traits<char>::eof()) {
*this->pptr() = std::char_traits<char>::to_char_type(c);
this->pbump(1);
}
return this->pubsync()? std::char_traits<char>::eof(): std::char_traits<char>::not_eof(c);
}
int sync() {
char* out(0);
int size(0);
Encode(this->pbase(), this->pptr() - this->pbase(), &out, size);
this->d_sbuf->sputn(out, size);
delete[] out; // dunno: it seems the output buffer is allocated but how?
this->setp(this->pbase(), this->epptr());
return this->d_sbuf->pubsync();
}
};
int main() {
encryptbuf sbuf(std::cout.rdbuf());
std::ostream eout(&sbuf);
eout << "print something encoded to standard output\n" << std::flush;
}
Now, creating an output operator for your records just printing to an std::ostream can be used to create an encoded
It's probably easiest to serialize your structure into a string, then encrypt the string. For example:
std::ostringstream buffer;
buffer << a_record.name << "\n" << a_record.location << "\n" << a_record.salary;
encode(buffer.str().c_str(), buffer.str().length(), /* ... */);
If it were me, I'd probably write encode (or at least a wrapper for it) to take input (and probably produce output) in a vector, string, or stream though.
If you want to get ambitious, there are other possibilities. First of all, #MooingDuck raises a good point that it's often worthwhile to overload operator<< for the class, instead of working with the individual items all the time. This will typically be a small function similar to what's above:
std::ostream &operator<<(std::ostream &os, record const &r) {
return os << r.name << "\n" << r.location << "\n" << r.salary;
}
Using this, you'd just have:
std::ostringstream os;
os << a_record;
encode(os.str().c_str(), os.str().length(), /* ... */);
Second, if you want to get really ambitious, you can put the encryption into (for one example) a codecvt facet, so you can automatically encrypt all the data as you write it to a stream, and decrypt it as you read it back in. Another possibility is to build the encryption into a filtering streambuf object instead. The codecvt facet is probably the method that should theoretically be preferred, but the streambuf is almost certainly easier to implement, with less unrelated "stuff" involved.