Variable sized packet - c++

I am trying to define a packet whose length is determined during an ns-3 simulation (think of it as a packet sent on the downlink containing schedule information whose length depends on the number of nodes in the network which can join/leave the network during simulation). Does anyone have any idea how I could approach this?

The traditional solution is to send the length first, followed by the data:
+------------+---------------------+
| uint32_t n | n - 4 bytes of data |
+------------+---------------------+
To decode, read the first four bytes, and then use the value in those bytes to determine how much more data there is.

The simplest introduction on how to do this is http://www.nsnam.org/support/faq/miscellaneous/#table
In practice, if you want to extend this code to store a variable-sized data structure, you can do this:
class MyHeader : public Header
{
public:
// new methods
void AppendData (uint16_t data);
std::vector<uint16_t> GetData (void) const;
static TypeId GetTypeId (void);
// overridden from Header
virtual uint32_t GetSerializedSize (void) const;
virtual void Serialize (Buffer::Iterator start) const;
virtual uint32_t Deserialize (Buffer::Iterator start);
virtual void Print (std::ostream &os) const;
private:
std::vector<uint16_t> m_data;
};
I will skip the obvious GetData/AppendData methods. Instead, we can focus on the Serialize/Deserialize methods:
uint32_t
MyHeader::GetSerializedSize (void) const
{
// two bytes of data to store
return m_data.size() * 2;
}
void
MyHeader::Serialize (Buffer::Iterator start) const
{
start.WriteHtonU32(GetSerializedSize());
for (std::vector<uint16_t>::const_iterator i = m_data.begin(); i != m_data.end(); i++)
{
start.WriteHtonU16 (*i);
}
}
uint32_t
MyHeader::Deserialize (Buffer::Iterator start)
{
uint32_t len = start.ReadNtohU32 ();
for (uint32_t i = 0; i < len; i++) {
m_data.append(start.ReadNtohU16())
}
return 4+len*2;
}

Related

Why is this binary output code causing a memory leak

I am new to C++ and I am trying to output a wave file in. I have had success with binary files in both C# and Java but I am not yet comfortable with C++ yet. I know about that arrays and objects should generally be created on the heap.
It is fine with the strings and the first getter
Whenever it gets to the second getter for the base class it runs out of memory.
This class is called waveWriter and it extends a class called WaveFormat that contains the getters
WaveWriter header:
class WaveWriter : WaveFormat {
private:
std::string fileName;
public:
WaveWriter(uint16_t channels, uint32_t sampleRate,
uint16_t bitsPerSample, double* lSampleData,
uint32_t lSampleLength, double* rSampleData,
uint32_t rSampleLength, bool isFloat, std::string outputName);
~WaveWriter();
void writeWave() {
std::ofstream myFile;
myFile = std::ofstream(fileName, std::ios::out | std::ios::binary);
// write the header
// sGroupID
myFile << S_GROUP_ID_HEADER;
// dwfilelength
myFile.write(reinterpret_cast<const char *> (GetDwFileLength()),
sizeof (GetDwFileLength()));
// sRiffType
myFile << S_RIFF_TYPE;
// write the format
// sGroupID
myFile << S_GROUP_ID_FORMAT;
// dwChunkSize
myFile.write(reinterpret_cast<const char *> (GetDwFormatChunkSize()),
sizeof (GetDwFormatChunkSize()));
// wFormatTag
........ blah blah blah
// close file
myFile.close();
return;
}
};
cpp for this class:
WaveWriter::WaveWriter(uint16_t channels, uint32_t sampleRate,
uint16_t bitsPerSample, double* lSampleData,
uint32_t lSampleLength, double* rSampleData,
uint32_t rSampleLength, bool isFloat, std::string outputName) :
WaveFormat(channels, sampleRate, bitsPerSample,
lSampleData, lSampleLength, rSampleData,
rSampleLength, isFloat) {
outputName.append(".wav");
this->fileName = outputName;
}
WaveWriter::~WaveWriter() {
this->~WaveFormat();
}
Header for WaveFormat contains private variables a constructor and getters like these to access the private variables:
public:
uint16_t GetCbSize() {
return cbSize;
}
uint32_t GetDwAvgBytesPerSec() {
return dwAvgBytesPerSec;
}
uint32_t GetDwChannelMask() {
return dwChannelMask;
}......
This is speculation based on your functions name, but I think this code:
myFile.write(reinterpret_cast<const char *> (GetDwFileLength()),sizeof (GetDwFileLength()));
is incorrect. Assuming GetDwFileLength() return size as value, it is incorrect to cast it to const char *. You need to save it in another argument and post its address to cast. Something like this:
auto val = GetDwFileLength();
myFile.write(reinterpret_cast<const char *> (&val), sizeof (val));
I see similar mistake several times in your code. This mistake can make invalid memory access.
In addition you should use virtual base destructor rather than calling base destructor from derived class. Never call base destructor in derived class.

Parsing custom data packets in an object oriented manner

I am currently developing some software in C++ where I am sending and receiving custom data packets. I want to parse and manage these packets in a well structured manner. Obviously I am first receiving the header and after that the body of the data. The main problem is that I don't like creating a Packet-Object with only the header information and later on adding the body data. What is an elegant way of parsing and storing custom data packets?
Here is a rough sketch of what such a custom data packet could look like:
+-------+---------+---------+----------+------+
| Magic | Command | Options | Bodysize | Body |
+-------+---------+---------+----------+------+
(Lets assume Magic is 4 bytes, Command 1 byte, Options 2 bytes, Bodysize 4 bytes and the body itself is variable in length.)
How would I parse this without using any third party libraries?
Normally I'd say something like this could be done to store packet data:
#include <array>
class Packet {
public:
explicit Packet(std::array<char, 10> headerbytes);
void set_body(std::vector<char> data);
std::vector<char> get_body();
int8_t get_command();
int16_t get_options();
bool is_valid();
private:
bool valid;
int8_t _command;
int16_t _options;
int32_t body_size;
std::vector<char> _data;
};
The problem is that I provide the header-information first and than add the body data in a hacky way later on. The packet object has a point of time where it is accessible in an incomplete state.
I first receive the header and after the header was received another receive call is made to read the body.
Would it make sense to have a parser instance that populates information into the packet object only make it accessible once it holds all needed information? Would it make sense to have a separate class for the header and the body? What would be the best design choice?
I am developing with C++ and for the sending and receiving of data over sockets the boost library is used.
If you don’t want to tie the data reading into one complete constructor (for understandable reasons of separation of concerns), this is a good application for non-polymorphic inheritance:
struct Header {
static constexpr SIZE=10;
Header(std::array<char,SIZE>);
std::int8_t get_command() const {return command;}
std::int16_t get_options() const {return options;}
std::int32_t body_size() const {return length;}
private:
std::int8_t command;
std::int16_t options;
std::int32_t length;
};
struct Packet : private Header {
using Body=std::vector<char>;
Packet(const Header &h,Body b) : Header(h),body(std::move(b))
{if(body.size()!=body_size()) throw …;}
using Header::get_command;
using Header::get_options;
const Body& get_body() const {return body;}
private:
Body body;
};
// For some suitable Stream class:
Header read1(Stream &s)
{return {s.read<Header::SIZE>()};}
Packet read2(const Header &h,Stream &s)
{return {h,s.read(h.body_size())};}
Packet read(Stream &s)
{return read2(read1(s),s);}
Note that the private inheritance prevents undefined behavior from deleting a Packet via a Header*, as well as the surely-unintended
const Packet p=read(s);
const Packet q=read2(p,s); // same header?!
Composition would of course work as well, but might result in more adapter code in a full implementation.
If you were really optimizing, you could make a HeaderOnly without the body size and derive Header and Packet from that.
For this case I would use the pipeline design pattern creating 3 packet processor classes:
Command (handles magic bytes too)
Options
Body (handles body size too)
all derived from one base class.
typedef unsigned char byte;
namespace Packet
{
namespace Processor
{
namespace Field
{
class Item
{
public:
/// Returns true when the field was fully processed, false otherwise.
virtual bool operator () (const byte*& begin, const byte* const end) = 0;
};
class Command: public Item
{
public:
virtual bool operator () (const byte*& begin, const byte* const end);
};
class Options: public Item
{
public:
virtual bool operator () (const byte*& begin, const byte* const end);
};
class Body: public Item
{
public:
virtual bool operator () (const byte*& begin, const byte* const end);
};
}
class Manager
{
public:
/// Called every time new data is received
void operator () (const byte* begin, const byte* const end)
{
while((*fields[index])(begin, end))
{
incrementIndex();
}
}
protected:
void incrementIndex();
Field::Command command;
Field::Options options;
Field::Body body;
Field::Item* const fields[3] = { &command, &options, &body };
byte index;
};
}
}
You can use exceptions to prevent creation of incomplete packet objects.
I'd use char pointers instead of vectors for performance.
// not intended to be inherited
class Packet final {
public:
Packet(const char* data, unsigned int data_len) {
if(data_len < header_len) {
throw std::invalid_argument("data too small");
}
const char* dataIter = data;
if(!check_validity(dataIter)) {
throw std::invalid_argument("invalid magic word");
}
dataIter += sizeof(magic);
memcpy(&command, dataIter, sizeof(command)); // can use cast & assignment, too
dataIter += sizeof(command);
memcpy(&options, dataIter, sizeof(options)); // can use cast & assignment, too
dataIter += sizeof(options);
memcpy(&body_size, dataIter, sizeof(body_size)); // can use cast & assignment, too
dataIter += sizeof(body_size);
if( data_len < body_size+header_len) {
throw std::invalid_argument("data body too small");
}
body = new char[body_size];
memcpy(body, dataIter, body_size);
}
~Packet() {
delete[] body;
}
int8_t get_command() const {
return command;
}
int16_t get_options() const {
return options;
}
int32_t get_body_size() const {
return body_size;
}
const char* get_body() const {
return body;
}
private:
// assumes len enough, may add param in_len for robustness
static bool check_validity(const char* in_magic) {
return ( 0 == memcmp(magic, in_magic, sizeof(magic)) );
}
constexpr static char magic[] = {'a','b','c','d'};
int8_t command;
int16_t options;
int32_t body_size;
char* body;
constexpr static unsigned int header_len = sizeof(magic) + sizeof(command)
+ sizeof(options) + sizeof(body_size);
};
Note: this is my first post in SO, so please let me know if something's wrong with the post, thanks.
I'm guessing you are trying Object-oriented networking. If so, the best solution for such parsing would be Flatbuffers or Cap’n Proto C++ code generator. By defining a schema, you will get state machine code that will parse the packets in an efficient and safe way.

Trying to access source device from boost::iostreams

I wrote a custom source device that counts the bytes read so far:
class socket_stream_source : public boost::iostreams::source
{
public:
int readSoFar=0;
socket_stream_source(socket_ptr sock) : _sock(sock)
{
}
std::streamsize read(char* s, std::streamsize n)
{
int readCount = _sock->read_some(boost::asio::buffer(s, n));
readSoFar += readCount;
return readCount;
}
private:
socket_ptr _sock;
};
I'm using it like this:
boost::iostreams::stream<socket_stream_source> in(sock);
How can I access my readSoFar variable ?
Or is there another way to count the bytes read so far from an istream ?
Just use the device access operators provided by boost::iostreams::stream, i.e.
T& operator*();
T* operator->();
In your code this suffice:
in->readSoFar;

Should C++ classes contain or be constructed/populated from protocol buffer messages

For protocol buffers in C++, I am wondering if it is better to contain a protobuf message in my class, or to have it be constructed from and populate an external protobuf message.
I could not find examples describing best practices for this case. I'm particular worried about performance differences between the two designs.
In my processing, I will have some cases where I am going to read only a few fields from my message and then route the message to another process (possibly manipulating the message before sendind it back out), and other cases where my objects will have a long lifetime and be used many times before being serialized again. In the first case, I could likely operate directly on the protobuf message and not even need my class, execpt to fit into an existing interface.
Here is an example message:
package example;
message Example {
optional string name = 1;
optional uint32 source = 2;
optional uint32 destination = 3;
optional uint32 value_1 = 4;
optional uint32 value_2 = 5;
optional uint32 value_3 = 6;
}
I could see one of the following designs for my class. I know these classes aren't doing anything else but accessing data, but that's not what I'm trying to focus on for this question.
Composition
class Widget
{
public:
Widget() : message_() {}
Widget(const example::Example& other_message)
: message_(other_message) {}
const example::Example& getMessage() const
{ return message_; }
void populateMessage(example::Example& message) const
{ message = message_; }
// Some example inspectors filled out...
std::string getName() const
{ return message_.name(); }
uint32_t getSource() const;
{ return message_.source(); }
uint32_t getDestination() const;
uint32_t getValue1() const;
uint32_t getValue2() const;
uint32_t getValue3() const;
// Some example mutators filled out...
void setName(const std::string& new_name)
{ message_.set_name(new_name); }
void setSource(uint32_t new_source);
{ message_.set_source(new_source); }
void setDestination(uint32_t new_destination);
void setValue1(uint32_t new_value);
void setValue2(uint32_t new_value);
void setValue3(uint32_t new_value);
private:
example::Example message_;
};
Standard data members
class Widget
{
public:
Widget();
Widget(const example::Example& other_message)
: name_(other_message.name()),
source_(other_message.source()),
destination_(other_message.destination()),
value_1_(other_messsage.value_1()),
value_2_(other_messsage.value_2()),
value_3_(other_messsage.value_3())
{}
example::Example getMessage() const
{
example::Example message;
populateMessage(message);
return message;
}
void populateMessage(example::Example& message) const
{
message.set_name(name_);
message.set_source(source_);
message.set_value_1(value_1_);
message.set_value_2(value_2_);
message.set_value_3(value_3_);
}
// Some example inspectors filled out...
std::string getName() const
{ return name_; }
uint32_t getSource() const;
{ return source_; }
uint32_t getDestination() const;
uint32_t getValue1() const;
uint32_t getValue2() const;
uint32_t getValue3() const;
// Some example mutators filled out...
void setName(const std::string& new_name)
{ name_ = new_name; }
void setSource(uint32_t new_source);
{ source_ = new_source; }
void setDestination(uint32_t new_destination);
void setValue1(uint32_t new_value);
void setValue2(uint32_t new_value);
void setValue3(uint32_t new_value);
private:
std::string name_;
uint32_t source_;
uint32_t destination_;
uint32_t value_1_;
uint32_t value_2_;
uint32_t value_3_;
};
There is no recognized "best practice" here. I have seen plenty of examples of both, and even written programs that worked both ways. Some people have very strong opinions about this, but in my opinion it depends on the use case. For example, as you say, if you plan to forward most of the data to another server, then it makes a lot of sense to keep the protobuf object around. But other times you have a more convenient internal representation -- for example, before protobufs added native support for maps, if you had a protobuf that represented a map as a repeated list of key/value pairs, you might want to convert it to an std::map upfront.

C++: compiler cannot read array of objects

I have a class that contains an array of object pointers as its member variable. I'm currently having an issue in getting the compiler to copy an object to the end of the array as when I step through the program the array of objects reads that its memory cannot be read. Anyone know what the issue might be?
void Notifications::operator+=(const iMessage& src) {
iMessage** temp2 = nullptr;
temp2 = new iMessage*[size+1];
if (size != 0){
for (int i = 0; i < size; i++) {
*temp2[i] = *messages[i];
}
}
*temp2[size] = src; //compiler states that it cannot read the data from temp2 after this point
delete[]messages;
for (int i = 0; i < size + 1; i++) {
*messages[i] = *temp2[i]; //Unhandled exception at 0x00C58F99 in w5.exe: 0xC0000005: Access violation reading location 0x00000000.
}
size++;
}
Notifications.h
#include "iMessage.h"
#include <vector>
namespace w5 {
class Notifications {
int size;
iMessage **messages;
public:
Notifications();
Notifications(const Notifications&);
Notifications& operator=(const Notifications&);
Notifications(Notifications&&);
Notifications&& operator=(Notifications&&);
~Notifications();
void operator+=(const iMessage&);
void display(std::ostream&) const;
};
}
IMessage.h
#ifndef _I_MESSAGE_H_
#define _I_MESSAGE_H_
// Workshop 5 - Containers
// iMessage.h
#include <iostream>
#include <fstream>
namespace w5 {
class iMessage {
public:
virtual void display(std::ostream&) const = 0;
virtual iMessage* clone() const = 0;
virtual bool empty() const = 0;
};
iMessage* getMessage(std::ifstream&, char);
}
#endif
Message.h
#include "iMessage.h"
namespace w5{
class Twitter : public iMessage {
std::string msg;
public:
Twitter(char, std::ifstream&);
virtual void display(std::ostream&) const;
virtual iMessage* clone() const;
virtual bool empty() const;
};
class Email : public iMessage {
std::string msg;
public:
Email(char, std::ifstream&);
virtual void display(std::ostream&) const;
virtual iMessage* clone() const;
virtual bool empty() const;
};
}
1) Just use vector.
2) You should always post exact compiler messages. "compiler states that it cannot read the data from temp2 after this point" is not good enough.
3) You allocate an array of pointers, and then dereference those pointers, but you never let the pointers point anywhere.
4) You delete the messages array and then proceed to copy back into it as if it was still there. (What you actually want to do is just assign messages = temp2.)
5) You're slicing objects all over the place, by using assignment to attempt to copy iMessage objects. There's a reason iMessage has a clone() function.
First you do
delete[]messages;
then you do
*messages[i] = *temp2[i];
attempting to access the array you've just deleted. I think you just want to take the pointer to the array you've just created:
messages = temp2;
You also do
*temp2[size] = src;
when temp2[size] doesn't point to anything. That should probably be
temp2[size] = src.clone();
to make a persistent copy of the argument and store it in the array.
It's rather tricky to follow this weird pointer-juggling; I think you also want to delete each element of messages before messages itself to avoid leaks. Why not just use std::vector to take care of memory allocation for you? That will reduce the whole insane dance to
std::vector<std::unique_ptr<iMessage>> messages;
void operator+=(const iMessage & src) {
messages.emplace_back(src.clone());
}
Also, _I_MESSAGE_H_ is a reserved name. You should remove the leading underscore.
You want to convert a const reference into a non-const pointer.
I wonder that the compiler doesn't throw errors. Which compiler you use?
Is something like this not possible?
void Notifications::operator+=(iMessage* src) {
I was not testing but this should also work:
void Notifications::operator+=(iMessage& src) {
*bar[foo] = &src;