C++: Populate a struct with data from a buffer - c++

I was wondering if I could have some recommendations on how to take data from a buffer and load them into a struct. For example, I have dealing with a DNS response buffer. I need to populate a DNS answer struct so that I can interpret the data. So far, I have the following:
int DugHelp::getPacket() {
memset(buf, 0, 2000); // clearing the buffer to make sure no "garbage" is there
if (( n = read(sock, buf, 2000)) < 0 {
exit(-1);
}
// trying to populate the DNS_Answers struct
dnsAnswer = (struct DNS_Answer *) & buf;
. . .
}
This is the struct that I have defined:
struct DNS_Answer{
unsigned char name [255];
struct {
unsigned short type;
unsigned short _class;
unsigned int ttl;
unsigned in len;
} types;
unsigned char data [2000];
};

It depends on the data format of buf. If the format is same with DNS_Answer. You can use memcpy. If their formats are same, you should align the bytes first.
#pragma pack (1)
struct DNS_Answer{
unsigned char name [255];
struct {
unsigned short type;
unsigned short _class;
unsigned int ttl;
unsigned in len;
} types;
unsigned char data [2000];
};
#pragma pop(1)
Then,
memcpy(dnsAnswer, buf, sizeof(DNS_Answer));
If their data formats aren't same, you have to parse them by yourself, or you can use DFDL (A data format descripation language.)

I do something a bit like this (rather untested) code:
Library Code:
namespace net {
using byte = unsigned char;
enum class endian
{
#ifdef _WIN32
little = 0,
big = 1,
native = little
#else
little = __ORDER_LITTLE_ENDIAN__,
big = __ORDER_BIG_ENDIAN__,
native = __BYTE_ORDER__,
#endif
};
constexpr bool is_little_endian()
{
return endian::native == endian::little;
}
template<typename POD>
byte* write_to_buffer(POD const& pod, byte* pos)
{
if(is_little_endian())
std::reverse_copy((byte*)&pod, (byte*)& pod + sizeof(pod), pos);
else
std::copy((byte*)&pod, (byte*)& pod + sizeof(pod), pos);
return pos + sizeof(pod);
}
template<typename POD>
byte const* read_from_buffer(byte const* pos, POD& pod)
{
if(is_little_endian())
std::copy(pos, pos + sizeof(pod), (byte*)&pod);
else
std::reverse_copy(pos, pos + sizeof(pod), (byte*)&pod);
return pos + sizeof(pod);
}
} // namespace net
Application Code:
struct DNS_Answer{
unsigned char name [255];
struct {
unsigned short type;
unsigned short _class;
unsigned int ttl;
unsigned int len;
} types;
unsigned char data [2000];
};
net::byte* write_to_buffer(DNS_Answer const& ans, net::byte* buf)
{
auto pos = buf;
pos = net::write_to_buffer(ans.name, pos);
pos = net::write_to_buffer(ans.types.type, pos);
pos = net::write_to_buffer(ans.types._class, pos);
pos = net::write_to_buffer(ans.types.ttl, pos);
pos = net::write_to_buffer(ans.types.len, pos);
pos = net::write_to_buffer(ans.data, pos);
return pos;
}
net::byte const* read_from_buffer(net::byte const* buf, DNS_Answer& ans)
{
auto pos = buf;
pos = net::read_from_buffer(pos, ans.name);
pos = net::read_from_buffer(pos, ans.types.type);
pos = net::read_from_buffer(pos, ans.types._class);
pos = net::read_from_buffer(pos, ans.types.ttl);
pos = net::read_from_buffer(pos, ans.types.len);
pos = net::read_from_buffer(pos, ans.data);
return pos;
}
This should be pretty portable, deals with different byte orders and avoids potential alignment problems. You can also transfer non-pod types by breaking them down into several POD pieces and sending those separately. For example std::string can be sent as a std::size_t for the length and the rest as a char array.

Related

Cast buffer to pointer Structure

I have two structure
struct TbtStreamHeader
{
short MsgLen;
short StreamId;
int SeqNo;
TbtStreamHeader()
{
MsgLen = StreamId = 0;
SeqNo = 0;
}
};
struct Multicast_OrderMsg
{
char MsgType; ///'N', 'X', 'M'
long long Timestamp;
double OrderId;
int Token;
char OrderType;
int Price;
int Quantity;
string ToString()
{
std::stringstream ss;
ss<<MsgType<<'|'<<Timestamp<<'|'<<(long long)OrderId<<"|0|"<<Token<<'|'<<OrderType<<'|'<<Price<<'|'<<Quantity;
return ss.str();
}
};
I am reading a file using ifstream, filling that file into the structure object because file is not into the format of structure, then copy the structure into char buffer. Here is the code for better understanding.
TbtStreamHeader hdr;
Multicast_OrderMsg oMsg;
hdr.MsgLen = 38;
hdr.StreamId = streamID;
hdr.SeqNo = 0;
bool firstLine = true;
while(ifs1)
{
if (!getline(ifs1, str1)) break;
istringstream ss(str1);
v1.clear();
while(ss)
{
string s1;
if(!getline( ss, s1, '|' ))
{
break;
}
v1.push_back(s1);
}
if(firstLine)
{
firstLine = false;
seq = atoi(v1[1].c_str());
hdr.SeqNo = seq;
getline(ifs1, str1);
}
else
{
char tempData[38];
int tempBufOffset=0;
hdr.SeqNo++;
oMsg.OrderId = atoll(v1[0].c_str());
oMsg.Token = atoi(v1[1].c_str());
oMsg.OrderType = v1[2][0];
oMsg.Price = atoi(v1[3].c_str());
oMsg.Quantity = atoi(v1[4].c_str());
oMsg.MsgType = 'N';
memcpy(tempData, &hdr, 8);
tempBufOffset += 8;
memcpy(tempData+tempBufOffset, oMsg, 30);
Multicast_OrderMsg* oMsgT = (Multicast_OrderMsg*)tempData+8; //Not able to caste
cout<<oMsgT->ToString()<<endl;
tempBufOffset += 30;
mcastTbt.streams[0].RecoveryPacket(tempData);
}
}
Here cout<< showing me segmentation fault. I am not able to understanding what wrong it is..?
At the location tempData+8 are no Multicast_OrderMsg objects, dereferencing such pointer is undefined behaviour as it violates the strict aliasing rule.
But that on itself does not usually cause seg. faults, the reason is tempData is not big enough to store the aliased Multicast_OrderMsg object. On most implementation the types require to be aligned at least to their size, which causes padding:
struct Multicast_OrderMsg
{
char MsgType; // 1
// Padding 7
long long Timestamp;// 8
double OrderId;// 8
int Token;// 4
char OrderType;// 1
// Padding 3
int Price;// 4
int Quantity;// 4
};
Making the structure 40 bytes long, you only have space for 30. So the printing tries to access memory locations not belonging to you, hence the segfault.

How to serialize structure data in C++?

I was asked in an interview to serialize data (so it could be stored in a buffer and sent over some network). This is what I came up with -
struct AMG_ANGLES {
float yaw;
float pitch;
float roll;
};
char b[sizeof(struct AMG_ANGLES)];
char* encode(struct AMG_ANGLES *a)
{
std::memcpy(b, &a, sizeof(struct AMG_ANGLES));
return b;
}
void decode(char* data)
{
// check endianess
AMG_ANGLES *tmp; //Re-make the struct
std::memcpy(&tmp, data, sizeof(tmp));
}
Is this correct? Can anyone give alternate designs? I did not get callback so I'm just trying to learn what I could have improved.
Is this correct?
Most likely, no.
The point of serialization is to convert the data into a form that is completely platform independent - e.g. does not rely on things like endianess or if a float is an IEEE 754 or something very different. This requires:
a) strict agreement on the intended format - e.g. if it's some kind of text (XML, JSON, CSV, ...) or if it's "raw binary" with explicit definitions of the meaning of each individual byte (e.g. like maybe "byte 1 is always the lowest 8 bits of the significand").
b) correct conversion to whatever the intended format is (e.g. maybe like ensuring that byte 1 is always the lowest 8 bits of the significand regardless of any/all platform differences)
However; it is at least technically possible that the code is not supposed to be portable and the specification ("agreement on the intended format") happens to match what you ended up with for the only platform that the code is designed for; and therefore it's at least technically possible that the code is correct.
There could be lots of improvements, but instead of telling all of them I suggest you to examine into cereal . It is widely used serialization/deserialization library, so lots of keypoints are thought.
Some of my thoughts are :
Your code depends on hardware which the program running on because of alignment and endianness. So the serialized data is not portable and compiler dependant.
char* encode(struct AMG_ANGLES *a) function returns char*, it is possibly leaked. To prevent the issue, let std::unique_ptr<T> decide its lifetime or wrap it with a class. But get rid of pointers somehow.
Templatize your serialize/deserialize operations. Otherwise, you could write same functions for other types.
template<typename T>
char* encode( T* a ) // I leave signature as is, just to demonstrate
{
std::memcpy( b , &a , sizeof(T) );
return b;
}
If the format is up to you, it is better to prefer human readable ones rather than binary archiving such as JSON, XML
can someone give alternate design in C?
The "standard" way would be to use printf and scanf to create an ascii representation of the data:
#include <limits.h>
#include <math.h>
#include <stdio.h>
#include <assert.h>
#include <float.h>
struct AMG_ANGLES {
float yaw;
float pitch;
float roll;
};
// declare a buffer at least this long to be sure encode works properly
#define AMG_ANGLES_BUFSIZE ( \
3 * ( /* 3 floats */ \
2 + /* digit and dot */ \
FLT_DECIMAL_DIG - 1 + /* digits after dot */ \
4 /* the 'e±dd' part */ \
) \
+ 2 /* spaces */ \
+ 1 /* zero terminating character */ \
)
int encode(char *dest, size_t destsize, const struct AMG_ANGLES *a) {
return snprintf(dest, destsize, "%.*e %.*e %.*e",
FLT_DECIMAL_DIG - 1, a->yaw,
FLT_DECIMAL_DIG - 1, a->pitch,
FLT_DECIMAL_DIG - 1, a->roll);
// my pedantic self wants to add `assert(snprintf_ret < AMG_ANGLES_BUFSIZE);`
}
int decode(struct AMG_ANGLES *dest, const char *data) {
return sscanf(data, "%e %e %e", &dest->yaw, &dest->pitch, &dest->roll) == 3 ? 0 : -1;
}
int main() {
char buf[AMG_ANGLES_BUFSIZE];
const struct AMG_ANGLES a = { FLT_MIN, FLT_MAX, FLT_MIN };
encode(buf, sizeof(buf), &a);
struct AMG_ANGLES b;
const int decoderet = decode(&b, buf);
assert(decoderet == 0);
assert(b.yaw == FLT_MIN);
assert(b.pitch == FLT_MAX);
assert(b.roll == FLT_MIN);
}
However in bare-metal embedded I try not to use scanf - it's a big function with some dependencies. So it's better to call strtof itself, but it needs some thinking:
int decode2(struct AMG_ANGLES *dest, const char *data) {
errno = 0;
char *endptr = NULL;
dest->yaw = strtof(data, &endptr);
if (errno != 0 || endptr == data) return -1;
if (*endptr != ' ') return -1;
data = endptr + 1;
dest->pitch = strtof(data, &endptr);
if (errno != 0 || endptr == data) return -1;
if (*endptr != ' ') return -1;
data = endptr + 1;
dest->roll = strtof(data, &endptr);
if (errno != 0 || endptr == data) return -1;
if (*endptr != '\0') return -1;
return 0;
}
or with removed code duplication:
int decode2(struct AMG_ANGLES *dest, const char *data) {
// array of pointers to floats to fill
float * const dests[] = { &dest->yaw, &dest->pitch, &dest->roll };
const size_t dests_cnt = sizeof(dests)/sizeof(*dests);
errno = 0;
for (int i = 0; i < dests_cnt; ++i) {
char *endptr = NULL;
*dests[i] = strtof(data, &endptr);
if (errno != 0 || endptr == data) return -1;
// space separates numbers, last number is followed by zero
const char should_be_char = i != dests_cnt - 1 ? ' ' : '\0';
if (*endptr != should_be_char) return -1;
data = endptr + 1;
}
return 0;
}
I needed to use some google and re-read chux answers to properly recall how to use FLT_DECIMAL_DIG in printf to print floats, that's most probably because I rarely worked with floats.
Keep in mind that, when using memcpy different architectures and different compilers will apply padding and endianness differently. To prevent the padding of the struct you could use an attribute provided by GCC
__attribute__ ((packed))
Nevertheless, this does not protect you from alternating endiannes.
The code for serializing and deserializing using memcpy might look like this:
#include <memory>
#include <cstring>
struct __attribute__((packed)) AMG_ANGLES {
float yaw;
float pitch;
float roll;
};
//The buffer is expected to be the same size as the T
template<typename T>
int serialize(const T &data,const std::unique_ptr<char[]> &buffer){
std::memcpy(buffer.get(), &data, sizeof(T));
return sizeof(T);
}
//The buffer is expected to be the same size as the ReturnType
template<typename ReturnType>
ReturnType deserialize(const std::unique_ptr<char[]> &buffer){
ReturnType tmp;
std::memcpy(&tmp, buffer.get(), sizeof(ReturnType));
return tmp;
}
int main()
{
struct AMG_ANGLES angles = {1.2, 1.3, 1.0};
std::unique_ptr<char[]> buffer(new char[sizeof(struct AMG_ANGLES)]);
int size = serialize(angles, buffer);
struct AMG_ANGLES angles_serialized = deserialize<AMG_ANGLES>(buffer);
}
It's better to make some kinds of class like std::stringstream..
std::stringstream is not good to save binary data but it works the same way you want.
so I could make some example that works with std::stringstream..
This code implement only for serialization but it also add code for deserialization.
// C++11
template < typename T, typename decltype(std::declval<T>().to_string())* = nullptr>
std::ostream& operator<< (std::ostream& stream, T&& val)
{
auto str = val.to_string();
std::operator <<(stream, str);
return stream;
}
struct AMG_ANGLES {
float yaw;
float pitch;
float roll;
std::string to_string() const
{
std::stringstream stream;
stream << yaw << pitch << roll;
return stream.str();
}
};
void Test()
{
std::stringstream stream;
stream << 3 << "Hello world" << AMG_ANGLES{1.f, 2.f, 3.f };
}

Sending pre-defined messages over TCP Sockets [duplicate]

I am creating a command-line client for minecraft. There is a full spec on the protocol that can be found here: http://mc.kev009.com/Protocol. To answer your question beforehand, yes I am a bit of a C++ noob.
I have various issues in implementing this protocol, of which each critical.
The protocol says that all types are big-endian. I have no idea how I should check whether my data is little-endian and if yes how to convert to big-endian.
The string datatype is a bit weird one. It's a modified UTF-8 string which is preceded by a short containing the string length. I have no idea how I should pack this into a simple char[] array nor how to convert my simple strings into modified UTF-8 ones.
Even if I knew how to convert my data to big-endian and create modified UTF-8 strings I still don't know how to pack this up into a char[] array and send this as a package. All I have done before is simple HTTP networking which is plain ASCII.
Explanations, links, related function names and short snippets much appreciated!
EDIT
1 and 3 is answered now. 1 is answered below by user470379. 3 is answered by this AWESOME thread that explains what I want to do very well: http://cboard.cprogramming.com/networking-device-communication/68196-sending-non-char*-data.html I'm not sure about the modified UTF-8 yet though.
A traditional approach is to define a C++ message structure for each protocol message and implement serialization and deserialization functions for it. For example Login Request can be represented like this:
#include <string>
#include <stdint.h>
struct LoginRequest
{
int32_t protocol_version;
std::string username;
std::string password;
int64_t map_seed;
int8_t dimension;
};
Now serialization functions are required. First it needs serialization functions for integers and strings, since these are the types of members in LoginRequest.
Integer serialization functions need to do conversions to and from big-endian representation. Since members of the message are copied to and from the buffer, the reversal of the byte order can be done while copying:
#include <boost/detail/endian.hpp>
#include <algorithm>
#ifdef BOOST_LITTLE_ENDIAN
inline void xcopy(void* dst, void const* src, size_t n)
{
char const* csrc = static_cast<char const*>(src);
std::reverse_copy(csrc, csrc + n, static_cast<char*>(dst));
}
#elif defined(BOOST_BIG_ENDIAN)
inline void xcopy(void* dst, void const* src, size_t n)
{
char const* csrc = static_cast<char const*>(src);
std::copy(csrc, csrc + n, static_cast<char*>(dst));
}
#endif
// serialize an integer in big-endian format
// returns one past the last written byte, or >buf_end if would overflow
template<class T>
typename boost::enable_if<boost::is_integral<T>, char*>::type serialize(T val, char* buf_beg, char* buf_end)
{
char* p = buf_beg + sizeof(T);
if(p <= buf_end)
xcopy(buf_beg, &val, sizeof(T));
return p;
}
// deserialize an integer from big-endian format
// returns one past the last written byte, or >buf_end if would underflow (incomplete message)
template<class T>
typename boost::enable_if<boost::is_integral<T>, char const*>::type deserialize(T& val, char const* buf_beg, char const* buf_end)
{
char const* p = buf_beg + sizeof(T);
if(p <= buf_end)
xcopy(&val, buf_beg, sizeof(T));
return p;
}
And for strings (handling modified UTF-8 the same way as asciiz strings):
// serialize a UTF-8 string
// returns one past the last written byte, or >buf_end if would overflow
char* serialize(std::string const& val, char* buf_beg, char* buf_end)
{
int16_t len = val.size();
buf_beg = serialize(len, buf_beg, buf_end);
char* p = buf_beg + len;
if(p <= buf_end)
memcpy(buf_beg, val.data(), len);
return p;
}
// deserialize a UTF-8 string
// returns one past the last written byte, or >buf_end if would underflow (incomplete message)
char const* deserialize(std::string& val, char const* buf_beg, char const* buf_end)
{
int16_t len;
buf_beg = deserialize(len, buf_beg, buf_end);
if(buf_beg > buf_end)
return buf_beg; // incomplete message
char const* p = buf_beg + len;
if(p <= buf_end)
val.assign(buf_beg, p);
return p;
}
And a couple of helper functors:
struct Serializer
{
template<class T>
char* operator()(T const& val, char* buf_beg, char* buf_end)
{
return serialize(val, buf_beg, buf_end);
}
};
struct Deserializer
{
template<class T>
char const* operator()(T& val, char const* buf_beg, char const* buf_end)
{
return deserialize(val, buf_beg, buf_end);
}
};
Now using these primitive functions we can readily serialize and deserialize LoginRequest message:
template<class Iterator, class Functor>
Iterator do_io(LoginRequest& msg, Iterator buf_beg, Iterator buf_end, Functor f)
{
buf_beg = f(msg.protocol_version, buf_beg, buf_end);
buf_beg = f(msg.username, buf_beg, buf_end);
buf_beg = f(msg.password, buf_beg, buf_end);
buf_beg = f(msg.map_seed, buf_beg, buf_end);
buf_beg = f(msg.dimension, buf_beg, buf_end);
return buf_beg;
}
char* serialize(LoginRequest const& msg, char* buf_beg, char* buf_end)
{
return do_io(const_cast<LoginRequest&>(msg), buf_beg, buf_end, Serializer());
}
char const* deserialize(LoginRequest& msg, char const* buf_beg, char const* buf_end)
{
return do_io(msg, buf_beg, buf_end, Deserializer());
}
Using the helper functors above and representing input/output buffers as char iterator ranges only one function template is required to do both serialization and deserialization of the message.
And putting all together, usage:
int main()
{
char buf[0x100];
char* buf_beg = buf;
char* buf_end = buf + sizeof buf;
LoginRequest msg;
char* msg_end_1 = serialize(msg, buf, buf_end);
if(msg_end_1 > buf_end)
; // more buffer space required to serialize the message
char const* msg_end_2 = deserialize(msg, buf_beg, buf_end);
if(msg_end_2 > buf_end)
; // incomplete message, more data required
}
For #1, you'll need to use ntohs and friends. Use the *s (short) versions for 16-bit integers, and the *l (long) versions for 32-bit integers. The hton* (host to network) will convert outgoing data to big-endian independently of the endianness of the platform you're on, and ntoh* (network to host) will convert incoming data back (again, independent of platform endianness)
off the top of my head...
const char* s; // the string you want to send
short len = strlen(s);
// allocate a buffer with enough room for the length info and the string
char* xfer = new char[ len + sizeof(short) ];
// copy the length info into the start of the buffer
// note: you need to hanle endian-ness of the short here.
memcpy(xfer, &len, sizeof(short));
// copy the string into the buffer
strncpy(xfer + sizeof(short), s, len);
// now xfer is the string you want to send across the wire.
// it starts with a short to identify its length.
// it is NOT null-terminated.

How to serialize a vector into an array of chars

I have a vector defined as:
std::vector<message *>
where message is:
struct message{
static unsigned int last_id;
unsigned int id;
std::string msg;
std::string timestamp;
}
My objective is to send this information using Winsock (from server to client), but this only allows sending chars as it appears in WinSock2.h. Taking this into account, I want to serialize all the information (id, msg and timestamp) in an array of chars in order to send it all together, and in the client, have a function to deserialize so as to have the same vector I had in the server.
How could I implement it?
Any help is appreciated.
The following offers a simple approach for the serialization problem.
However, note that it is not portable. It assumes same environment conditions on both sides (client/server), i.e. endianness and sizeof int and size_t. This assumption is probably unsatisfactory when writing server/client programs, and your code should handle this aspect as well.
For example, if you can say that 32 bits is a sufficient size for the id value and the length of your strings, you can use htonl when serializing, and ntohl when deserializing.
Serializer:
class MessageSerializer
{
public:
MessageSerializer(const message& messageStruct)
: m_msgRef(messageStruct)
, m_msgLength(m_msgRef.msg.length())
, m_timeLength(m_msgRef.timestamp.length())
{}
size_t RequiredBufferSize() const
{
return sizeof(int) + sizeof(size_t)*2 + m_msgLength + m_timeLength;
}
void Serialize(void* buffer) const
{
PushNum (buffer, m_msgRef.id);
PushString (buffer, m_msgRef.msg.c_str(), m_msgLength);
PushString (buffer, m_msgRef.timestamp.c_str(), m_timeLength);
}
private:
const message& m_msgRef;
const size_t m_msgLength;
const size_t m_timeLength;
template<typename INTEGER>
void PushNum(void*& buffer, INTEGER num) const
{
INTEGER* ptr = static_cast<INTEGER*>(buffer);
//copying content
*ptr = num;
//updating the buffer pointer to point the next position to copy
buffer = ++ptr;
}
void PushString(void*& buffer, const char* cstr, size_t length) const
{
PushNum(buffer, length);
//copying string content
memcpy(buffer, cstr, length);
//updating the buffer pointer to point the next position to copy
char* ptr = static_cast<char*>(buffer);
ptr += length;
buffer = ptr;
}
};
Deserializer:
class MessageDeserializer
{
public:
MessageDeserializer(const char* messageBuffer)
: m_msgBuffer(messageBuffer)
{}
void Deserialize(message& messageOut)
{
messageOut.id = PopNum<int>(m_msgBuffer);
messageOut.msg = PopString(m_msgBuffer);
messageOut.timestamp = PopString(m_msgBuffer);
}
private:
const void* m_msgBuffer;
template<typename INTEGER>
INTEGER PopNum(const void*& buffer) const
{
const INTEGER* ptr = static_cast<const INTEGER*>(buffer);
//copying content
INTEGER retVal = *ptr;
//updating the buffer pointer to point the next position to copy
buffer = ++ptr;
return retVal;
}
std::string PopString(const void*& buffer) const
{
size_t length = PopNum<size_t>(buffer);
const char* ptr = static_cast<const char*>(buffer);
//copying content
std::string retVal(ptr, length);
//updating the buffer pointer to point the next position to copy
ptr += length;
buffer = ptr;
return retVal;
}
};
Then your using code could be something like:
//...
MessageSerializer serializer(*myVector[i]);
char* buffer = new char[serializer.RequiredBufferSize()];
serializer.Serialize(buffer);
and:
//...
message myMsg;
MessageDeserializer(input).Deserialize(myMsg);
You can use the Boost serialization library to save/load your structure to an array of char. The boost library is widely used in C++, and if you're not familiar with it, I'd recommend taking a look at it.
Instead of using winsock, you could learn to use Boost sockets and make your C++ code work on almost any platform, instead of just Windows, but that's another topic.
Here's an example of how to serialize your vector, and recover it from the other side of the socket:
#include <vector>
#include <boost/serialization/nvp.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
struct message {
static unsigned int last_id;
unsigned int id;
std::string msg;
std::string timestamp;
template <class ArchiveT>
void serialize(ArchiveT& ar, const unsigned int /*version*/) // function used to serialize (save/load) data from the boost serialization library
{
ar & boost::serialization::make_nvp("LastId", last_id);
ar & boost::serialization::make_nvp("Id", id);
ar & boost::serialization::make_nvp("Msg", msg);
ar & boost::serialization::make_nvp("Timestamp", timestamp);
}
};
unsigned int message::last_id;
template <class T>
void serialize_save(const T& obj, std::string& outString)
{
std::stringstream binaryOut;
boost::archive::binary_oarchive outArchive(binaryOut);
outArchive << obj;
outString = binaryOut.str();
}
template <class T>
void serialize_load(T& dataOut, const void* data, const size_t dataSize)
{
const char* dataPtr = reinterpret_cast<const char*>(data);
std::string dataString(dataPtr, dataPtr + dataSize);
std::stringstream dataStream(dataString);
boost::archive::binary_iarchive binArchive(dataStream);
binArchive >> dataOut;
}
void init_vector(std::vector<message*>& vect) {
const size_t vectorSize = 2;
vect.resize(vectorSize);
for (size_t i = 0; i < vectorSize; i++) {
vect[i] = new message();
vect[i]->last_id = 0;
vect[i]->id = 1;
vect[i]->msg = "This is a message";
vect[i]->timestamp = "12:02pm";
}
}
int main() {
std::vector<message*> messages;
init_vector(messages); // initialize the vector. set it to any data
std::string outputBuffer;
serialize_save(messages, outputBuffer); // save the vector to a string (array of char)
socket_write(outputBuffer.c_str(), outputBuffer.size()); // write the serialized data to the socket
// on the reception side
std::string receiveBuffer;
socket_read(receiveBuffer); // receive socket data
std::vector<message*> receivedMessages;
serialize_load(receivedMessages, receiveBuffer.c_str(), receiveBuffer.size()); // from the array of character recover the vector
// here the vector receivedMessages contains the same values saved in init_vector()
}
You can change the export format if you'd like by changing the boost::archive::binary_iarchive object. For instance, replace it to boost::archive::xml_iarchive for serializing objects to XML. There are other formats provided by the library. Another advantage is that it supports versioning.

overcome rpc endianness convert

I want to assign unsigned char[8] to uint64 (c language) , pass this value with RPC and convert the uint64 back to unsigned char[8] with the same bytes order (cpp language).
The problem is that the RPC may convert my uint64 endianness.
what is the best way to do this?
While the endiannes may change, you can still extract individual bytes from uint64_t portably, e.g.:
void to_bytes(uint64_t from, char* to) {
for(size_t i = 0; i < sizeof from; ++i, from >>= 8)
to[i] = from & 0xff;
}
Alternatively, use reversing copy operations:
#ifdef BOOST_LITTLE_ENDIAN
inline void xcopy(void* dst, void const* src, size_t n)
{
char const* csrc = static_cast<char const*>(src);
std::reverse_copy(csrc, csrc + n, static_cast<char*>(dst));
}
#elif defined(BOOST_BIG_ENDIAN)
inline void xcopy(void* dst, void const* src, size_t n)
{
char const* csrc = static_cast<char const*>(src);
std::copy(csrc, csrc + n, static_cast<char*>(dst));
}
#endif
void to_bytes(uint64_t from, char* to) {
xcopy(to, &from, sizeof from);
}
void from_bytes(char const* from, uint64_t* to) {
xcopy(to, from, sizeof *to);
}
unit8_t data[8];
// fill the array, then ...
uint64_t carrier = data [0];
size_t position;
for (position = 1; position < 8; ++position) {
carrier <<= 8;
carrier |= data[position];
}
// ... on the other end
// variables of same type
position = 8;
while (position--) {
data[position] = 0xFF & carrier;
carrier >>= 8;
}
This should do it, since the value (so you don't have to worry about endianness) of carrier will be (hopefully) correctly transmitted by the RPC protocol.
Note the use of uint8_t instead of char. The later isn't guaranteed to be 1/8th of uint64_t.
The code should have well defined behaviour for both C and C++. For C++ you should rather use std::array instead of a raw array.