I want to serialize my protocol buffer to a char*. Is this possible? I know one can serialize to file as per:
fstream output("/home/eamorr/test.bin", ios::out | ios::trunc | ios::binary);
if (!address_book.SerializeToOstream(&output)) {
cerr << "Failed to write address book." << endl;
return -1;
}
But I'd like to serialize to a C-style char* for transmission across a network.
How to do this? Please bear in mind that I'm very new to C++.
That's easy:
size_t size = address_book.ByteSizeLong();
void *buffer = malloc(size);
address_book.SerializeToArray(buffer, size);
Check documentation of MessageLite class also, it's parent class of Message and it contains useful methods.
You can serailze the output to a ostringstream and use stream.str() to get the string and then access the c-string with string.c_str().
std::ostringstream stream;
address_book.SerializeToOstream(&stream);
string text = stream.str();
char* ctext = text.c_str();
Don't forget to include sstream for std::ostringstream.
You can use ByteSizeLong() to get the number of bytes the message will occupy and then SerializeToArray() to populate an array with the encoded message.
Solution with a smart pointer for the array:
size_t size = address_book.ByteSizeLong();
std::unique_ptr<char[]> serialized(new char[size]);
address_book.SerializeToArray(&serialized[0], static_cast<int>(size));
Still one more line of code to take care of the fact that the serialized data can contain zero's.
std::ostringstream stream;
address_book.SerializeToOstream(&stream);
string text = stream.str();
char* ctext = text.c_str(); // ptr to serialized data buffer
// strlen(ctext) would wrongly stop at the 1st 0 it encounters.
int data_len = text.size(); // length of serialized data
Related
I serialize the file via the code beneath, and send it over winsocks, this works fine with textfiles, but when I tried to send a jpg, the string contains \0 as some of the character elements, so the sockets only send part of the string, thinking \0 is the end, i was considering replacing \0 with something else, but say i replace it with 'xx', then replace it back on the other end, what if the file had natural occurrences of 'xx' that get lost? Sure I could make a large, unlikely sequence, but that bloats the file.
Any help appreciated.
char* read_file(string path, int& len)
{
std::ifstream infile(path);
infile.seekg(0, infile.end);
size_t length = infile.tellg();
infile.seekg(0, infile.beg);
len = length;
char* buffer = new char[len]();
infile.read(buffer, length);
return buffer;
}
string load_to_buffer(string file)
{
char* img;
int ln;
img = read_file(file, ln);
string s = "";
for (int i = 1; i <= ln; i++){
char c = *(img + i);
s += c;
}
return s;
}
Probably somewhere in your code (that isn't seen in the code you have posted) you use strlen() or std::string::length() to send the data, and/or you use std::string::c_str() to get the buffer. This results in truncated data because these functions stop at \0.
std::string is not good to handle binary data. Use std::vector<char> instead, and remove the new[] stuff.
I'm serializing data into binary file using ofstream/ifstream. Data is divided in 2 vectors of strings, one for data names and other for data values, std::vector<std::string> dataNames, std::vector<std::string> dataValues.
I'm writting the data using this function:
void Data::SaveData(std::string path)
{
std::ofstream outfile(path, std::ofstream::binary);
outfile.write(reinterpret_cast<const char *>(&dataNames[0]), dataNames.size() * sizeof(std::string));
outfile.write(reinterpret_cast<const char *>(&dataValues[0]), dataValues.size() * sizeof(std::string));
outfile.close();
}
And reading it using:
bool Data::LoadData(std::string path)
{
bool ret = false;
std::ifstream file(path, std::ifstream::in | std::ifstream::binary);
if (file.is_open())
{
// get length of file:
file.seekg(0, file.end);
int length = file.tellg();
file.seekg(0, file.beg);
char * buffer = new char[length];
file.read(buffer, length);
if (file)
{
char* cursor = buffer;
uint32_t bytes = length / 2;
dataNames.resize(bytes / sizeof(std::string));
memcpy(dataNames.data(), cursor, bytes);
cursor += bytes;
dataValues.resize(bytes / sizeof(std::string));
memcpy(dataValues.data(), cursor, bytes);
delete[] buffer;
buffer = nullptr;
}
file.close();
ret = true;
}
return ret;
}
It works. I can write and read it correctly. Except if any of the strings in dataNames or dataValues has 16 chars or more.
Example of data using strings with less than 16 chars:
dataNames[0] = "Type"
dataNames[1] = "GameObjectCount"
dataValues[0] = "Scene"
dataValues[1] = "5"
data 15 chars
Example of data using strings with more than 16 chars:
dataNames[0] = "Type"
dataNames[1] = "GameObjectsCount" //Added a s. Now have 16 chars
dataValues[0] = "Scene"
dataValues[1] = "5"
data 16 chars
Here you can see that word "GameObjectsCount" doesn't appear and extrange characters are shown.
When reading this file the string is not valid. Sometimes it's empty, sometimes says "Error reading characters of string", sometimes is a radom letter...
Any idea?
Reinterpreting a vector in the manner you have above isn't correct.
outfile.write(reinterpret_cast<const char *>(&dataNames[0]), dataNames.size() * sizeof(std::string));
You don't know how the vector stores data (on the heap, etc..), and you can't assume that you can blindly cast the pointer and write whatever you see out to a file as a method to serialize the data. Furthermore, a std::string isn't necessarily an in-place character array of the size of the input. It's more likely a pointer to an object on the heap.
So, if you want to serialize the data in a vector or another stdlib type, you'll need to write a function to do that manually by iterating over the items and writing them in a properly delimited way.
Im having problems writing string into a binary file. This is my code:
ofstream outfile("myfile.txt", ofstream::binary);
std::string text = "Text";
outfile.write((char*) &text, sizeof (string));
outfile.close();
Then, I try to read it,
char* buffer = (char*) malloc(sizeof(string));
ifstream infile("myfile.txt", ifstream::binary);
infile.read(buffer, sizeof (prueba));
std::string* elem = (string*) buffer;
cout << *elem;
infile.close();
I just cant get it to work. I am sorry, I am just desperate. Thank you!
To write a std::string to a binary file, you need to save the string length first:
std::string str("whatever");
size_t size=str.size();
outfile.write(&size,sizeof(size));
outfile.write(&str[0],size);
To read it in, reverse the process, resizing the string first so you will have enough space:
std::string str;
size_t size;
infile.read(&size, sizeof(size));
str.resize(size);
infile.read(&str[0], size);
Because strings have a variable size, unless you put that size in the file you will not be able to retrieve it correctly. You could rely on the '\0' marker that is guaranteed to be at the end of a c-string or the equivalent string::c_str() call, but that is not a good idea because
you have to read in the string character by character checking for the null
a std::string can legitimately contain a null byte (although it really shouldn't because calls to c_str() are then confusing).
the line
outfile.write((char*) &text, sizeof (string));
is not correct
sizeof(string) doesn't return the length of the string, it returns the sizeof the string type in bytes.
also do not cast text to char* using a C cast, you can get hold of the char* by using the appropriate member function text.c_str()
you can simply write
outfile << text;
instead.
Why are you using pointers to std::string class?
You should not use sizeof with std::string, as it returns the size of the std::string object, and not the real size of the string inside.
You should try:
string text = "Text";
outfile.write(text.c_str(), text.size());
or
outfile << text;
Should probably also use c_str() to get the char pointer too, instead of that straight crazy cast.
Your code is wrong wrong way you are using to write & read the file
and file extension error you are trying to read text file .txt
correct code
Write to file
std::string text = "Text";
ofstream outfile("myfile.dat", ofstream::binary | ios::out);
outfile.write(&text,sizeof (string));//can take type
outfile.write(&text,sizeof (text));//can take variable name
outfile.close();
reading file
char* buffer = (char*) malloc(sizeof(string));
ifstream infile("myfile.dat", ifstream::binary | ios::in);
infile.read(buffer, sizeof (prueba));
std::string* elem = (string*) buffer;
cout << *elem;
infile.close();
Try This it will work
I had the same problem. I found the perfect answer here: Write file in binary format
Key issues: use string::length to get the length of the string when writing out and use resize() before reading the string. And both for reading and writing, use mystring.c_str() instead the string itself.
Try this code snippet.
/* writing string into a binary file */
fstream ifs;
ifs.open ("c:/filename.exe", fstream::binary | fstream::in | fstream::out);
if (ifs.is_open())
{
ifs.write("string to binary", strlen("string to binary"));
ifs.close();
}
Here is a good example.
The reader and writer
#include<string>
#include<fstream>
#include<memory>
class BinarySearchFile{
BinarySearchFile::BinarySearchFile(std::string file_name){
// concatenate extension to fileName
file_name += ".dat";
// form complete table data filename
data_file_name = file_name;
// create or reopen table data file for reading and writing
binary_search_file.open(data_file_name, std::ios::binary); // create file
if(!binary_search_file.is_open()){
binary_search_file.clear();
binary_search_file.open(data_file_name, std::ios::out | std::ios::binary);
binary_search_file.close();
binary_search_file.open(data_file_name), std::ios::out | std::ios::in | std::ios::binary | std::ios::ate;
}
std::fstream binary_search_file;
void BinarySearchFile::writeT(std::string attribute){
if(binary_search_file){
binary_search_file.write(reinterpret_cast<char *>(&attribute), attribute.length() * 2);
}
}
std::string BinarySearchFile::readT(long filePointerLocation, long sizeOfData)
{
if(binary_search_file){
std::string data;
data.resize(sizeOfData);
binary_search_file.seekp(filePointerLocation);
binary_search_file.seekg(filePointerLocation);
binary_search_file.read(&data[0], sizeOfData);
return data;
}
};
The reader call
while (true){
std::unique_ptr<BinarySearchFile> data_file(new BinarySearchFile("classroom.dat"));
std::string attribute_value = data_file->read_data(0, 20);
}
The writer call
data_file->write_data("packard ");
The writer writes a total of 50 bytes
"packard 101 500 "
The reader is to read the first 20 bytes and the result is "X packard X" where X represents some malformed bytes of data. Why is the data read back in x-number of bytes corrupt?
You can't simply write data out by casting it's address to a char* and hoping to get anything useful. You have to define the binary format you want to use, and implement it. In the case of std::string, this may mean outputing the length in some format, then the actual data. Or in the case where fixed length fields are needed, forcing the string (or a copy of the string) to that length using std::string::resize, then outputting that, using std::string::data() to get your char const*.
Reading will, of course, be similar. You'll read the data into a std::vector<char> (or for fixed length fields, a char[]), and parse it.
binary_search_file.write(reinterpret_cast<char *>(&attribute), attribute.length() * 2);
It is incorrect to cast std::string to char* if you need char* you must use attribute.c_str().
std::string apart from string pointer contains other data members, for example, allocator, your code will write all of that data to file. Also I don't see any reason to multiply string length by 2. +1 makes sense if you want to output terminating zero.
None of the posted answers I've read work, so I'm asking again.
I'm trying to copy the string data pointed to by a char pointer into a char array.
I have a function that reads from a ifstream into a char array
char* FileReader::getNextBytes(int numberOfBytes) {
char *buf = new char[numberOfBytes];
file.read(buf, numberOfBytes);
return buf;
}
I then have a struct :
struct Packet {
char data[MAX_DATA_SIZE]; // can hold file name or data
} packet;
I want to copy what is returned from getNextBytes(MAX_DATA_SIZE) into packet.data;
EDIT: Let me show you what I'm getting with all the answers gotten below (memcpy, strcpy, passing as parameter). I'm thinking the error comes from somewhere else. I'm reading a file as binary (it's a png). I'll loop while the fstream is good() and read from the fstream into the buf (which might be the data array). I want to see the length of what I've read :
cout << strlen(packet.data) << endl;
This returns different sizes every time:
8
529
60
46
358
66
156
After that, apparently there are no bytes left to read although the file is 13K + bytes long.
This can be done using standard library function memcpy, which is declared in / :
strcpy(packet.data, buf);
This requires file.read returns proper char series that ends with '\0'. You might also want to ensure numberOfBytes is big enough to accommodate the whole string. Otherwise you could possibly get segmentation fault.
//if buf not properly null terminated added a null char at the end
buf[numberofbytes] = "\0"
//copy the string from buf to struc
strcpy(packet.data, buf);
//or
strncpy(packet.data, buf);
Edit:
Whether or not this is being handled as a string is a very important distinction. In your question, you referred to it as a "string", which is what got us all confused.
Without any library assistance:
char result = reader.getNextBytes(MAX_DATA_SIZE);
for (int i = 0; i < MAX_DATA_SIZE; ++MAX_DATA_SIZE) {
packet.data[i] = result[i];
}
delete [] result;
Using #include <cstring>:
memcpy(packet.data, result, MAX_DATA_SIZE);
Or for extra credit, rewrite getNextBytes so it has an output parameter:
char* FileReader::getNextBytes(int numberOfBytes, char* buf) {
file.read(buf, numberOfBytes);
return buf;
}
Then it's just:
reader.getNextBytes(MAX_DATA_SIZE, packet.data);
Edit 2:
To get the length of a file:
file.seekg (0, ios::end);
int length = file.tellg();
file.seekg (0, ios::beg);
And with that in hand...
char* buffer = new char[length];
file.read(buffer, length);
Now you have the entire file in buffer.
strlen is not a valid way to determine the amount of binary data. strlen just reads until it finds '\0', nothing more. If you want to read a chunk of binary data, just use a std::vector, resize it to the amount of bytes you read from the file, and return it as value. Problem solved.