Tcp protocol prefix the length of message - c++

I'm trying to prefix the length of message so its 4 bytes and use big-endian in C++, but i don't understand how to do that. In Node.JS i'm doing it like that:
var buffer = new Buffer("HELLO WORLD");
//create a buffer with +4 bytes
var consolidatedBuffer = new Buffer(4 + buffer.length);
//write at the beginning of the buffer, the total size
consolidatedBuffer.writeInt32BE(buffer.length, 0);
//Copy the message buffer to the consolidated buffer at position 4 (after the 4 bytes about the size)
buffer.copy(consolidatedBuffer, 4);
i'm trying to achieve the same result in C++, but don't know how to do the same, can someone please show me how that is done in C++? the result should be string byte array
I'm doing that on Mac:
std::string data = "HELLO WORLD";
uint32_t length = htonl( data.length() );
std::string payload;
payload.append(std::to_string(length));
payload.append(data);

One way of doing this would be something like
std::vector<char> pack(const std::string& str) {
const uint32_t sz = str.size();
const uint32_t n_sz = htonl(sz);
std::vector<char> result(sizeof(sz) + sz);
memcpy(result.data(), &n_sz, sizeof(n_sz));
memcpy(result.data() + sizeof(sz), str.data(), sz);
return result;
}

Related

Converting binary data in string to char

So I have some data I convert from packet to string, in binary (datagram):
std::string Packet::packetToString()
{
//packing to one bitset
std::bitset<208> pak(std::string(std::bitset<2>(type).to_string() + std::bitset<64>(num1).to_string() + std::bitset<64>(num2).to_string() + std::bitset<64>(num3).to_string() + std::bitset<4>(state).to_string() + std::bitset<4>(id).to_string() + "000000"));
std::string temp;
std::bitset<8> tempBitset(0);
for (int i = pak.size() - 1; i >= 0; i--)
{
tempBitset[i % 8] = pak[i];
if (i % 8 == 0)
{
char t = static_cast<char> (tempBitset.to_ulong());
temp.push_back(t);
}
}
return temp;
}
Then I want to convert this string to char array (in this case char buffer[26];) and send it with SendTo("127.0.0.1", 1111, buffer, 26);
What's the problem:
Packet pak1(... data I input ... );
string packet;
packet = pak1.packetToString();
char buffer[26];
strcpy_s(buffer, packet.c_str());
Data send with this array seems to be erased in case 0x00(NULL) appears. This is caused by c_str() i guess. How can I deal with this? :)
strcpy() and strcpy_s() copy null terminated c-strings. So indeed, if there's any 0x00 char in the c_str() the copy will end.
Use std::copy() to copy the full data, regardless of the any 0x00 that you might encounter:
copy (packet.begin(), packet.end(), buffer); // hoping packet.size()<26
or with copy_n():
copy_n (packet.begin(), 26, buffer); // assuming that packet is at least 26 bytes

How to get BLOB data using oracle ODBC

I'm trying to get image that are stored in BLOB and then save it as jpg.
Here i retrieve the binary data and save it in str;
string str;
SQLCHAR buf[500] = {0};
while ((SQL_SUCCEEDED(SQLGetData(StmtHandle, colnum, SQL_C_BINARY, buf, sizeof(buf), NULL))))
{
string data(reinterpret_cast< const char* >(buf), reinterpret_cast< const char* >(buf) + sizeof(buf));
str = str + data;
}
Then i write it in the file
ofstream file;
file.open("C:\\Users\\tom\\Desktop\\img.jpeg");
file << str;
file.close();
and i get the incorrect image.
What's wrong with this method of data extraction (i used this) ?
I'm not familiar with ODBC programming, but at first sight, one issue I can see is that you assume your data length is multiple of your buffer size. But the last read is not guaranteed to return exactly 500 bytes of data.
You should write something like that. Maybe:
string str;
SQLCHAR buf[500];
SQLLEN cbLeft; // #bytes remained
while ((SQL_SUCCEEDED(SQLGetData(StmtHandle,
colnum,
SQL_C_BINARY,
buf,
sizeof(buf),
&cbLeft))))
// ^^^^^^^
{
string data(reinterpret_cast< const char* >(buf),
reinterpret_cast< const char* >(buf)
+ cbLeft);
// ^^^^^^
str = str + data;
Please take a few minutes to review Using Length/Indicator Values in order to check how the length/indicator value is used.

C++ linux socket that sends array of strings(char**) in a single call as concatenated string

I got array of lengthy strings. I have to send them as single concatenated string in a send (int __fd, const void *__buf, size_t __n, int __flags) call. I am afraid its a CPU consuming process to construct the single concatenated string(char*). Is there a facility to send array of strings tail to head?
I don't want to call send more than once for single meaningful string as long as recv fires more than once at the receiving end.
I wonder why there ain't a standardized extendable string structure like linked list in C/C++ so that readers can jump on to next buffer at the end of a buffer. I wish atleast std::string implement this.
You don't need to concatenate all strings in one go. This won't be very CPU consuming since it will happen underneath anyways, but it may or may not consume a lot of memory.
If you are using flags in send then you should determine the socket buffer size. Concatenate your strings up to that buffer size and then send them one buffer at a time
void send_strings(int sockfd, char ** strings, size_t numstrings, int flags) {
// get the socket write buffer size
int buflen;
unsigned int m = sizeof(bufsize);
if(getsockopt(sockfd,SOL_SOCKET,SO_SNDBUF,(void *)&buflen, &m)) {
perror("getsockopt"); return; }
char buffer[buflen];
int bufsize = 0;
while (numstrings--) {
char * string = *(strings++);
size_t length = strlen(string);
// if the string would exceed the buffer
while (length > buflen - bufsize) {
memcpy(buffer + bufsize, string, buflen - bufsize);
length -= buflen - bufsize;
string += buflen - bufsize;
// send a full buffer
send(sockfd, buffer, buflen, flags);
bufsize = 0;
}
// copy the string into the buffer
memcpy(buffer + bufsize, string, length);
bufsize += length;
}
// send the rest
if (bufsize) {
send(sockfd, buffer, bufsize, flags);
}
}
http://linux.die.net/man/2/send states the use of MSG_MORE flag. When set, it will initiate transmission only after the send call without MSG_MORE flag. So call send for each data chunk with MSG_MORE except for the last data chunk.

How to give dynamic array to google::protobuf::io::ArrayOutputStream?

I have the following method that serialize my protocol buffer message and this works perfectly :
string DroolsMsgTransmission::serialize(const google::protobuf::Message* msg, const HeaderMsg& header)const
{
unsigned char buffer[20000];
google::protobuf::io::ArrayOutputStream arr(buffer, sizeof(buffer));
google::protobuf::io::CodedOutputStream output(&arr);
output.WriteVarint32(header.ByteSize());
header.SerializeToCodedStream(&output);
output.WriteVarint32(msg->ByteSize());
msg->SerializeToCodedStream(&output);
return string((char*)buffer, output.ByteCount());
}
Is it possible to use a dynamic buffer instead? I tried the following:
string DroolsMsgTransmission::serialize(const google::protobuf::Message* msg, const HeaderMsg& header)const
{
char* buffer = new char[header.ByteSize() + msg->ByteSize()]();
google::protobuf::io::ArrayOutputStream arr(buffer, sizeof(buffer));
google::protobuf::io::CodedOutputStream output(&arr);
output.WriteVarint32(header.ByteSize());
header.SerializeToCodedStream(&output);
output.WriteVarint32(msg->ByteSize());
msg->SerializeToCodedStream(&output);
string str = string(buffer);
delete buffer;
return str;
}
But this is not working. When I am trying the line above, the returned string does not contain the serialized data at all.
I also tried to use OstreamOutputStream instead of ArrayOutputStream, but no luck.
EDIT
Thanks to comments, I almost made it working :
string DroolsMsgTransmission::serialize(const google::protobuf::Message* msg, const HeaderMsg& header)const
{
char* buffer = new char[header.ByteSize() + msg->ByteSize()]();
google::protobuf::io::ArrayOutputStream arr(buffer, header.ByteSize() + msg->ByteSize());
google::protobuf::io::CodedOutputStream output(&arr);
output.WriteVarint32(header.ByteSize());
header.SerializeToCodedStream(&output);
output.WriteVarint32(msg->ByteSize());
msg->SerializeToCodedStream(&output);
string str = string(buffer ,output.ByteCount());
//return string((char*)buffer, output.ByteCount());
int toto = header.ByteSize() + msg->ByteSize();
int tata = output.ByteCount();
int titi = sizeof(buffer);
delete buffer;
return str;
}
What I did, I replaced this line
google::protobuf::io::ArrayOutputStream arr(buffer, sizeof(buffer));
by this line
google::protobuf::io::ArrayOutputStream arr(buffer, header.ByteSize() + msg->ByteSize());
I it better now, but I still have a problem where the returned string seems a little bit tuncated at the end. It might be related with WriteVarint32, but I don't understand. Somebody can explain why?
Thank you.
You're writing 4 things (the size of the header, the header, the size of the message, the message) but only allocating space for two of them (the header and message).
Hint: a Varint32 never takes more than 5 bytes.
(Also: you need the size in two places - when allocating the buffer and constructing arr. Compute it once and store it in a local variable.)

How to read x characters from position y in a char * buffer?

I am reading through a buffer (char *) and i have a cursor, where i am tracking my starting position of the buffer, is there a way to copy characters 7-64 out of the buffer, or is my best bet to just loop the buffer from poistion x to position y?
The size of the destination buffer is the result of another function dynamically computed
Initializing this returns
variable-sized object 'version' may not be initialized
Relevant code parts:
int32_t size = this->getObjectSizeForMarker(cursor, length, buffer);
cursor = cursor + 8; //advance cursor past marker and size
char version[size] = this->getObjectForSizeAndCursor(size, cursor, buffer);
-
char* FileReader::getObjectForSizeAndCursor(int32_t size, int cursor, char *buffer) {
char destination[size];
memcpy(destination, buffer + cursor, size);
}
-
int32_t FileReader::getObjectSizeForMarker(int cursor, int eof, char * buffer) {
//skip the marker and read next 4 byes
cursor = cursor + 4; //skip marker and read 4
unsigned char *ptr = (unsigned char *)buffer + cursor;
int32_t objSize = (ptr[0] << 24) | (ptr[1] << 16) | (ptr[2] << 8) | ptr[3];
return objSize;
}
Move the pointer to buffer six units ahead (to get to the seventh index), and then memcpy 64-7 (57) bytes, e.g.:
const char *buffer = "foo bar baz...";
char destination[SOME_MAX_LENGTH];
memcpy(destination, buffer + 6, 64-7);
You may want to terminate the destination array so that you can work with it using standard C string functions. Note that we're adding the null character at the 58th index, after the 57 bytes that were copied over:
/* terminate the destination string at the 58th byte, if desired */
destination[64-7] = '\0';
If you need to work with a dynamically sized destination, use a pointer instead of an array:
const char *buffer = "foo bar baz...";
char *destination = NULL;
/* note we do not multiply by sizeof(char), which is unnecessary */
/* we should cast the result, if we're in C++ */
destination = (char *) malloc(58);
/* error checking */
if (!destination) {
fprintf(stderr, "ERROR: Could not allocate space for destination\n");
return EXIT_FAILURE;
}
/* copy bytes and terminate */
memcpy(destination, buffer + 6, 57);
*(destination + 57) = '\0';
...
/* don't forget to free malloc'ed variables at the end of your program, to prevent memory leaks */
free(destination);
Honestly, if you're in C++, you should really probably be using the C++ strings library and std::string class. Then you can call the substr substring method on your string instance to get the 57-character substring of interest. It would involve fewer headaches and less re-inventing the wheel.
But the above code should be useful for both C and C++ applications.