I have a C++ struct for an update packet to be exchanged between servers, and another struct to implement information about neighbors, and a vector of struct neighbor is inside update packet.
struct neighbor;
struct update_packet {
uint16_t num_update_fields;
uint16_t port;
uint32_t IP;
vector<struct neighbor> neighbors;
update_packet(char * IPstr, int port) :
num_update_fields(num_nodes),
IP(IP_to_int(IPstr)), port((uint16_t) port)
{ };
};
struct neighbor {
uint32_t IP;
uint16_t port;
int16_t nil;
uint16_t server_id;
uint16_t cost;
neighbor(char * IPstr, uint16_t port, uint16_t server_id, uint16_t cost) :
IP(IP_to_int(IPstr)), port(port), nil(0),
server_id(server_id), cost(cost) { };
};
I want to exchange this struct in general message format (like IP datagram, for example) through UDP sockets and read the information in the message on the receiving end.
How can I achieve this? Is my design for the structs a poor design for my purpose?
What you're asking about is serialization. At it's simplest, if you have a POD type and know that the machines at both ends are the same type you can just cast and send:
struct Foo {
uint32_t a;
uint16_t p;
};
Foo f { 1, 2 };
sendto(targetFD, reinterpret_cast<const char*>(&f), sizeof(f), 0);
Your structure couldn't be used this way because of the vector. The data in a vector is dynamically allocated so it isn't laid out with the rest of the structure. If you need a variable length field like this, an array (or a std::array) along with an indicator of the number of elements would be useful.
Casting structures like this is fast but not really portable. If you want to be able to pass to other languages or platforms you would prefer a serialization format. Google's Protocol Buffers (https://code.google.com/p/protobuf/) and several other serialization libraries are designed to allow this.
Slightly more c++ style serialization and deserialization is desribed in this article - https://rodgert.github.io/2014/09/09/type-driven-wire-protocols-with-boost-fusion-pt1/
Related
I have been implementing a communication protocol in C++ and I have decided to model one packet in below given manner.
union control_pkt_u{
struct pkt_parts_t{
uint8_t header[3]; // Control packet header
uint8_t payload[NO_PYLD_BYTES_IN_CONTROL_PACKET]; // Control packet payload
};
uint8_t pkt_array[NO_BYTES_IN_PACKET];
};
As soon as I need to access to the elements of the union
pkt.pkt_parts_t.header[0] = APP_MSG_DEB;
I receive an error during compilation:
invalid use of struct Manager::control_pkt_u::pkt_parts_t
Please can anybody tell me what I am doing wrong?
Because you are just defining a struct in your control_pkt_u union and it is just a declaration, it is not initialised when you create an object from it. You need to declare it as a member like this and reach your member pkt_parts_.
union control_pkt_u {
struct pkt_parts_t {
uint8_t header[3]; // Control packet header
uint8_t payload[NO_PYLD_BYTES_IN_CONTROL_PACKET]; // Control packet payload
} pkt_parts_;
uint8_t pkt_array[NO_BYTES_IN_PACKET];
};
pkt.pkt_parts_.header[0] = APP_MSG_DEB;
You can change the struct definiation to this by using Anonymous structure:
struct {
uint8_t header[3]; // Control packet header
uint8_t payload[NO_PYLD_BYTES_IN_CONTROL_PACKET]; // Control packet payload
} pkt_parts_t;
Then you don't need to change other code.
The problem in a nutshell: I'm considering a rewrite of some packet handling code for a UDP protocol ... from C to C++. Obviously, I can just use the structures that I had with C to model the data, but then what would be the point? So, for the sake of argument, we have:
struct { uint8:2 type; uint8:6 con_id; uint16 seq_num } packet_header;
struct { packet_header ph; uint16 command; } control_packet;
struct { packet_header ph; uchar8 data[DATASIZE]; } data_packet;
In this case, "type" is 0 for data, 1 for control, 2 for encrypted data and 3 for encrypted control. There would also be structs reprenting each of the control packets, for argument, lets say "open" and "close".
Now... I would normally have a type that the function that fetches the packet returns and the function that sends accepts:
struct {
size_t pkt_len;
union {
uchar8 raw_packet[DATASIZE];
packet_header ph;
control_packet cp;
control_open_packet cop;
control_close_packet ccp;
data_packet dp;
} p;
} packet;
... this is all simplified for argument... but the point is: when sending a packet, we want to create a packet of a certain type and send a generic packet. Conversely, when receiving a packet, you want to receive a generic packet and and cast it to more specific types as you classify it.
So where's the question? My question is: if Variant and Any are not appropriate ways to express this, what are? Am I stuck with union in this case? Certainly I don't violate the stipulation of POD types (so far) for the union. I've been scouring both C++11 and Boost (and to some extent looking at C++14 ... although I'm stuck with C++11) and I don't see offered solutions.
You can use both union as it is in your example or class hierarchy with packet_header a the top and specific derived classes for each type of the packet:
struct packet_header { uint8:2 type; uint8:6 con_id; uint16 seq_num };
struct control_packet : packet_header { uint16 command; };
struct data_packet: packet_header { uchar8 data[DATASIZE]; };
struct packet
{
size_t pkt_len;
union
{
uchar8 raw_packet[DATASIZE];
packet_header ph;
control_packet cp;
control_open_packet cop;
control_close_packet ccp;
data_packet dp;
};
};
Maybe you need to seek for elegance and flexibility in the code that handles your packets. I doubt that defining data in some other way (using boost or using C++17 features) will lead to a considerable gain.
Note the anonymous union in the struct packet. It allows accessing fields of specific packet types directly. C++ style expects the name of the struct directly after the struct keyword.
Originally, We have two applications communication with TCP/IP, and both of them are implemented by C++. The messages between them are custom message type.
Now the client program will be changed to web application based on nodejs and the communication between them will be changed to message bus such as rabbitmq
So the message type between them should be changed.
JSON comes to my mind first, however the custom messages are too complicated, which are defined by template and inheritance. It seems that convert the custom message to JSON is not a good option. Am I right?
class Address {
int network;
int addressType;
//...
};
class MsgType{
unsigned char msgSeq;
unsigned int msgLen;
//...
};
class Message{
Address destination;
Address source;
MsgType msgType;
//...
};
template <typename T, int RESPONSE_TYPE>
class ResponseMessage : public Message{
//...
}
typedef struct{
int number;
int type;
}ConfigResp;
class CfgResp : public ResponseMessage<ConfigResp, CONFIG_REQUEST>
{
//...
}
Protocol Buffers is another option for me to do that. What should I do?
redefine the custom message into protocol buffer? no no
Here is my solution: Just wrap the whole original custom message (binary type) into protocol buffer as one message in the server side, then decode the custom message(binary type) in client side. Is that possible?
It looks like you are structuring your application to become more extensible. Not using a nice message format is completely counter to that aim.
Don't embed your binary format inside a protocol buffer chunk. You'll gain nothing - you'll need to rewrite parsing and writing code for each component that wants to use the message bus. Thats wasted time and effort.
There is pain in mapping your C++ structures to JSON or protocol buffers. But it will make hooking into those messages using node.js or other things peeking into the message bus much easier later.
Personally I'd use protocol buffers - since they're more type safe. And there are differences between handling of various types in JSON libraries, because the JSON format is (intentionally) lax. In particular I've found handling of long integers problematic.
Typically I write a helper template struct for each class I need to convert - then conversion becomes a lot of boilerplate. Something like
template<typename T> class ProtocolBufferHelper {
}
template<> class ProtocolBufferHelper<Address> {
typedef address_buffer protocol_buffer_type;
void writeToProtocolBuffer( const Address &a, address_buffer & buffer) {
buffer.setNetwork(a.network);
...
}
...
}
template<> class ProtocolBufferHelper<Message> {
void writeToProtocolBuffer( const Message &m, address_buffer & buffer) {
::writeToProtocolBuffer(buffer.getDestination(), m.destination);
::writeToProtocolBuffer(buffer.getSource(), m.source);
...
}
}
template<typename T> void writeToProtocolBuffer( const T &value, ProtocolBufferHelper<T>::protocol_buffer_type & buffer ) {
ProtocolBufferHelper<T>::writeToProtocolBuffer(value, buffer);
}
You'll have to forgive me for not remembering exactly what the protocol buffer syntax is in C++ (its been a while...). but hopefully its enough to get you started.
I'm currently working on a Client/Server application, sending packets via TCP.
To read the incoming packet, I do something like this:
struct SomeRandomStruct
{
int nVal1;
int nVal2;
};
SomeRandomStruct* pStruct = reinterpret_cast<SomeRandomStruct*>(pBuffer);
Right now there's a small problem. Let's say I have a struct like this
struct SomeNewStruct
{
int nNameLen;
int nPassLen;
char szName[];
char szPass[];
};
Since the size/length of szName & szPass is sent in the same packet, is there a way I can set its size ,,within the reinterpret_cast" or do I have to read the packet manually?
Friendly warning:
stop!
Consider something like google protocol buffers to encode messages for you in a cross-platform and safe way.
Here are some reasons why:
How big is an int? On your machine? today? tomorrow? It's undefined in the c++ standard.
What is the binary representation of an int? Is it the same on the machine sending and the machine receiving? Will it always be? (If you think 'yes' then you're wrong). Again the c++ standard has nothing to say on the matter.
What is the padding between data members in the struct? Is it the same for all compilers on all hosts? (answer: no)
ask yourself why functions like htons() exist. They are there because not all machines are the same, but the communication between them must be.
I could go on...
As your question is tagged C++, I'd propose you to do dynamic allocation in constructor and deallocation in destructor. That way, you could have simply pointers in your struct :
struct struct SomeNewStruct {
int nNameLen;
int nPassLen;
char *szName;
char *szPass;
SomeNewStruct(int nameLen , int passLen) {
// set and alloc...
}
SomeNewStruct(SomeNewStruct &src) {
// alloc and copy
}
~SomeNewStruct() {
// dealloc ...
}
}
You could even use std::string for szName and SzPass letting the STL deal with those low level allocation details :
struct struct SomeNewStruct {
int nNameLen;
int nPassLen;
std::string szName;
std::string szPass;
}
Is there a C++ (or MSVC) method of automatically padding a struct to a minimum size? For example, imagine the following pseudo-code:
#pragma pad(256) // bytes
struct SETUPDATA {
int var1;
double var2;
};
where sizeof(SETUPDATA) = 256 bytes
The goal here being, during development this struct's members can change without changing the footprint size at runtime.
You can use a union
struct SETUPDATA {
union { struct your_data; char [256]; }
}
or something like this. This ensures it's at least 256 but only as long as your_data is not larger.
You can also add a simple assert after that just does a compiler check assert(sizeof(struct SETUPDATA) == 256)
One way is to inherit from your "real" structure and use sizeof() to make up the padded structure, for example:
struct blah_real
{
int a;
};
struct blah : public blah_real
{
private:
char _pad[256 - sizeof(blah_real)];
};
You could use #ifdef DEBUG to only do this in the debug build and just use the real structure in release build.
The first thing you have ot ask yourself is why your application cares if the struct size changes. That indicate fragility to future changes and your design may be better served by instead allowing the application to seamlessly work in the face of struct size changes.
Perhaps you're trying to serialize the data directly and don't want to face changes in the format, but in that case you're already tying yourself to one specific representation of the structure in memory. For example support the size of one of the builtin-type members changes due to a compiler upgrade or options.
But let's say you really do want to do this.
Just wrap the data in an impl and pad the real struct:
struct SetupData
{
struct Impl
{
int var1;
double var2;
};
Impl impl_;
unsigned char pad_[256 - sizeof(Impl)];
};