I have the following method that serialize my protocol buffer message and this works perfectly :
string DroolsMsgTransmission::serialize(const google::protobuf::Message* msg, const HeaderMsg& header)const
{
unsigned char buffer[20000];
google::protobuf::io::ArrayOutputStream arr(buffer, sizeof(buffer));
google::protobuf::io::CodedOutputStream output(&arr);
output.WriteVarint32(header.ByteSize());
header.SerializeToCodedStream(&output);
output.WriteVarint32(msg->ByteSize());
msg->SerializeToCodedStream(&output);
return string((char*)buffer, output.ByteCount());
}
Is it possible to use a dynamic buffer instead? I tried the following:
string DroolsMsgTransmission::serialize(const google::protobuf::Message* msg, const HeaderMsg& header)const
{
char* buffer = new char[header.ByteSize() + msg->ByteSize()]();
google::protobuf::io::ArrayOutputStream arr(buffer, sizeof(buffer));
google::protobuf::io::CodedOutputStream output(&arr);
output.WriteVarint32(header.ByteSize());
header.SerializeToCodedStream(&output);
output.WriteVarint32(msg->ByteSize());
msg->SerializeToCodedStream(&output);
string str = string(buffer);
delete buffer;
return str;
}
But this is not working. When I am trying the line above, the returned string does not contain the serialized data at all.
I also tried to use OstreamOutputStream instead of ArrayOutputStream, but no luck.
EDIT
Thanks to comments, I almost made it working :
string DroolsMsgTransmission::serialize(const google::protobuf::Message* msg, const HeaderMsg& header)const
{
char* buffer = new char[header.ByteSize() + msg->ByteSize()]();
google::protobuf::io::ArrayOutputStream arr(buffer, header.ByteSize() + msg->ByteSize());
google::protobuf::io::CodedOutputStream output(&arr);
output.WriteVarint32(header.ByteSize());
header.SerializeToCodedStream(&output);
output.WriteVarint32(msg->ByteSize());
msg->SerializeToCodedStream(&output);
string str = string(buffer ,output.ByteCount());
//return string((char*)buffer, output.ByteCount());
int toto = header.ByteSize() + msg->ByteSize();
int tata = output.ByteCount();
int titi = sizeof(buffer);
delete buffer;
return str;
}
What I did, I replaced this line
google::protobuf::io::ArrayOutputStream arr(buffer, sizeof(buffer));
by this line
google::protobuf::io::ArrayOutputStream arr(buffer, header.ByteSize() + msg->ByteSize());
I it better now, but I still have a problem where the returned string seems a little bit tuncated at the end. It might be related with WriteVarint32, but I don't understand. Somebody can explain why?
Thank you.
You're writing 4 things (the size of the header, the header, the size of the message, the message) but only allocating space for two of them (the header and message).
Hint: a Varint32 never takes more than 5 bytes.
(Also: you need the size in two places - when allocating the buffer and constructing arr. Compute it once and store it in a local variable.)
Related
I'm trying to base64 decode a string, then convert that value to a char array for later use. The decode works fine, but then I get garbage data when converting.
Here's the code I have so far:
std::string encodedData = "VGVzdFN0cmluZw=="; //"TestString"
std::vector<BYTE> decodedData = base64_decode(encodedData);
char* decodedChar;
decodedChar = new char[decodedData.size() +1]; // +1 for the final 0
decodedChar[decodedData.size() + 1] = 0; // terminate the string
for (size_t i = 0; i < decodedData.size(); ++i) {
decodedChar[i] = decodedData[i];
}
vector<BYTE> is a typedef of unsigned char BYTE, as taken from this SO answer. The base64 code is also from this answer (the most upvoted answer, not the accepted answer).
When I run this code, I get the following value in the VisualStudio Text Visualiser:
TestStringÍ
I've also tried other conversion methods, such as:
char* decodedChar = reinterpret_cast< char *>(&decodedData[0]);
Which gives the following:
TestStringÍÍÍýýýýÝÝÝÝÝÝÝ*b4d“
Why am I getting the garbage data at the end of the string? What am i doing wrong?
EDIT: clarified which answer in the linked question I'm using
char* decodedChar;
decodedChar = new char[decodedData.size() +1]; // +1 for the final 0
Why would you manually allocate a buffer and then copy to it when you have std::string available that does this for you?
Just do:
std::string encodedData = "VGVzdFN0cmluZw=="; //"TestString"
std::vector<BYTE> decodedData = base64_decode(encodedData);
std::string decodedString { decodedData.begin(), decodedData.end() };
std::cout << decodedString << '\n';
If you need a char * out of this, just use .c_str()
const char* cstr = decodedString.c_str();
If you need to pass this on to a function that takes char* as input, for example:
void someFunc(char* data);
//...
//call site
someFunc( &decodedString[0] );
We have a TON of functions and abstractions and containers in C++ that were made to improve upon the C language, and so that programmers wouldn't have to write things by hand and make same mistakes every time they code. It would be best if we use those functionalities wherever we can to avoid raw loops or to do simple modifications like this.
You are writing beyond the last element of your allocated array, which can cause literally anything to happen (according to the C++ standard). You need decodedChar[decodedData.size()] = 0;
I serialize the file via the code beneath, and send it over winsocks, this works fine with textfiles, but when I tried to send a jpg, the string contains \0 as some of the character elements, so the sockets only send part of the string, thinking \0 is the end, i was considering replacing \0 with something else, but say i replace it with 'xx', then replace it back on the other end, what if the file had natural occurrences of 'xx' that get lost? Sure I could make a large, unlikely sequence, but that bloats the file.
Any help appreciated.
char* read_file(string path, int& len)
{
std::ifstream infile(path);
infile.seekg(0, infile.end);
size_t length = infile.tellg();
infile.seekg(0, infile.beg);
len = length;
char* buffer = new char[len]();
infile.read(buffer, length);
return buffer;
}
string load_to_buffer(string file)
{
char* img;
int ln;
img = read_file(file, ln);
string s = "";
for (int i = 1; i <= ln; i++){
char c = *(img + i);
s += c;
}
return s;
}
Probably somewhere in your code (that isn't seen in the code you have posted) you use strlen() or std::string::length() to send the data, and/or you use std::string::c_str() to get the buffer. This results in truncated data because these functions stop at \0.
std::string is not good to handle binary data. Use std::vector<char> instead, and remove the new[] stuff.
My goal is to get this:
BYTE Data1[] = {0x6b,0x65,0x79};
BYTE Data2[] = {0x6D,0x65,0x73,0x73,0x61,0x67,0x65};
But my starting point is:
std::string msg = "message";
std::string key = "key";
I am not able to get from std::string to BYTE[].
I tried the following:
std::vector<BYTE> msgbytebuffer(msg.begin(), msg.end());
BYTE* Data1 = &msgbytebuffer[0];
This didn't cause compile or run time error. However, the end result (I feed this to a winapi function - crypto api) was not the same as when I used the actual byte array like in top most ({0x6D,0x65,0x73,0x73,0x61,0x67,0x65}).
You can use string::c_str() function which returns a pointer to c style string that can be passed to winapi functions like:
foo(string.c_str());
What it actually does is that it returns a pointer to an array that contains a null-terminated sequence of characters.
I suppose BYTE[] is actually a char array. You can assign your std::string to char array by doing:
std::string str = "hello";
BYTE byte[6]; // null terminated string;
strcpy(byte, str.c_str()); // copy from str to byte[]
If you want to copy the str without the 0 at the end, use strncpy instead:
BYTE byte[5];
strncpy(byte, str.c_str(), str.length());
Seems me that winapi is waiting a null terminated c-string. You can achieve that by using:
msg.c_str();
or, using your BYTE type, something like that:
std::vector<BYTE> msgbytebuffer(msg.length() + 1, 0);
std::copy(msg.begin(), msg.end(), msgbytebuffer.begin());
I want to parse UTF-8 file to ustring, I read this file in str.
There is an error:
terminate called after throwing an instance of 'Glib::ConvertError'.
What should I do?
char* cs = (char*) malloc(sizeof(char) * str.length());
strcpy(cs, str.c_str());
ustring res;
while (strlen(cs) > 0) {
gunichar ch = g_utf8_get_char(cs);
res.push_back(ch);
cs = g_utf8_next_char(cs);
}
wofstream wout("output");
cout << res << endl;
This looks very wrong:
char* cs = (char*) malloc(sizeof(str.c_str()));
as sizeof(str.c_str()) is bound to give you some small number like 4 or 8 (whichever is the size of a pointer on your machine, as the result of str.c_str().
Of course, it doesn't REALLY matter, since the next line, you are leaking the memory you just allocated incorrectly:
cs = const_cast<char*> (str.c_str());
I'm far from convinced that you need the const_cast<char *> (it is certainly WRONG to do this, since modifying the string inside a string is undefined behaviour).
None of the posted answers I've read work, so I'm asking again.
I'm trying to copy the string data pointed to by a char pointer into a char array.
I have a function that reads from a ifstream into a char array
char* FileReader::getNextBytes(int numberOfBytes) {
char *buf = new char[numberOfBytes];
file.read(buf, numberOfBytes);
return buf;
}
I then have a struct :
struct Packet {
char data[MAX_DATA_SIZE]; // can hold file name or data
} packet;
I want to copy what is returned from getNextBytes(MAX_DATA_SIZE) into packet.data;
EDIT: Let me show you what I'm getting with all the answers gotten below (memcpy, strcpy, passing as parameter). I'm thinking the error comes from somewhere else. I'm reading a file as binary (it's a png). I'll loop while the fstream is good() and read from the fstream into the buf (which might be the data array). I want to see the length of what I've read :
cout << strlen(packet.data) << endl;
This returns different sizes every time:
8
529
60
46
358
66
156
After that, apparently there are no bytes left to read although the file is 13K + bytes long.
This can be done using standard library function memcpy, which is declared in / :
strcpy(packet.data, buf);
This requires file.read returns proper char series that ends with '\0'. You might also want to ensure numberOfBytes is big enough to accommodate the whole string. Otherwise you could possibly get segmentation fault.
//if buf not properly null terminated added a null char at the end
buf[numberofbytes] = "\0"
//copy the string from buf to struc
strcpy(packet.data, buf);
//or
strncpy(packet.data, buf);
Edit:
Whether or not this is being handled as a string is a very important distinction. In your question, you referred to it as a "string", which is what got us all confused.
Without any library assistance:
char result = reader.getNextBytes(MAX_DATA_SIZE);
for (int i = 0; i < MAX_DATA_SIZE; ++MAX_DATA_SIZE) {
packet.data[i] = result[i];
}
delete [] result;
Using #include <cstring>:
memcpy(packet.data, result, MAX_DATA_SIZE);
Or for extra credit, rewrite getNextBytes so it has an output parameter:
char* FileReader::getNextBytes(int numberOfBytes, char* buf) {
file.read(buf, numberOfBytes);
return buf;
}
Then it's just:
reader.getNextBytes(MAX_DATA_SIZE, packet.data);
Edit 2:
To get the length of a file:
file.seekg (0, ios::end);
int length = file.tellg();
file.seekg (0, ios::beg);
And with that in hand...
char* buffer = new char[length];
file.read(buffer, length);
Now you have the entire file in buffer.
strlen is not a valid way to determine the amount of binary data. strlen just reads until it finds '\0', nothing more. If you want to read a chunk of binary data, just use a std::vector, resize it to the amount of bytes you read from the file, and return it as value. Problem solved.