I am writing an C++ Application which reads several voltages from a device. I receive these measurements in an float[] and I want to send this array via UDP to a MATLAB-Script.
the C++-function sendto needs to get an char[] buffer and I really have no idea how to convert the float[] into a char[] buffer so i can reassemble it easily in MATLAB. Any Ideas?
Another problem i encountered is that line
addr.sin_addr = inet_addr("127.0.0.1");
inet_addr returns an unsigned long, but my compiler tells me that the = operator does not accept an unsigend long datatype on its right side. Any Iideas about this?
You can always treat any object variable as a sequence of bytes. For this very purpose, it is explicitly allowed (and does not violate aliasing or constitute type punning) to reinterpret any object pointer as a pointer to the first element in an array of bytes (i.e. any char type).
Example:
T x;
char const * p = reinterpret_cast<char const *>(&x);
for (std::size_t i = 0; i != sizeof x; ++i) { /* p[i] is the ith byte in x */ }
For your case:
float data[N];
char const * p = reinterpret_cast<char const *>(data);
write(fd, p, sizeof data);
Decide if you want to format the UDP messages as text or binary. If text, you can convert floats to strings using boost::lexical_cast. You can frame the string valus in the UDP message any way you want (comma separated values, newline separated, etc.), or you could use a known format such as JSON.
If you want to transmit binary data, select an known format, such as XDR which is used by ONC RPC and use existing library tools to create the binary messages.
As for the inet_addr error, addr.sin_addr is a struct in_addr. You need to assign the result to the s_addr member of the sin_addr struture like this:
addr.sin_addr.s_addr = inet_addr("127.0.0.1");
There are two questions in your post. I believe that is not how it's supposed to be.
As for float[]->byte[] conersion - you should check how matlab stores it's floating point variables. If, by any chance, it uses the same format as you compiler, for your computer setup etc etc only, you can simply send these as a byte array[]. In any other case - incompatible float byte format, multiple machines - you have to write a manual conversion. First each float to (for example) string, then many floats. Your line could look like:
1.41234;1.63756;456345.45634
As for the addr.sin_addr - I think you are doing it wrong. You should access
addr.sin_addr.s_addr = inet_addr("1.1.1.1");
Related
[EDIT]I wanted write uint64_t to char* array in network byte order to send it as UDP datagram with sendto, uint64_t has 8 bytes so I convert them as follow:
void strcat_number(uint64_t v, char* datagram) {
uint64_t net_order = htobe64(v);
for (uint8_t i=0; i<8 ;++i) {
strcat(datagram, (const char*)((uint8_t*)&net_order)[i]);
}
}
wchich give me
warning: cast to pointer from integer of different size [-Wint-to-pointer-xast]
strcat(datagram, (const char*)((uint8_t*)&net_order)[i]);
how can I get rid of this warning or maybe do this number converting simpler or clearer?
((uint8_t*)&net_order)
this is a pointer to net_order casted to a uint8_t pointer
((uint8_t*)&net_order)[i]
this is the i-th byte of the underlying representation of net_order.
(const char*)((uint8_t*)&net_order)[i]
this is the same as above, but brutally casted to a const char *. This is an invalid pointer, and it is what the compiler is warning you about; even just creating this pointer is undefined behavior, and using it in any way will almost surely result in a crash.
Notice that, even if you somehow managed to make this kludge work, strcat is still the wrong function, as it deals with NUL-terminated strings, while here you are trying to put binary data inside your buffer, and binary data can naturally contain embedded NULs. strcat will append at the first NUL (and stop at the first NUL in the second parameter) instead of at the "real" end.
If you are building a buffer of binary data you have to use straight memcpy, and most importantly you cannot use string-related functions that rely on the final NUL to know where the string ends, but you have to keep track explicitly of how many bytes you used (i.e. the current position in the datagram).
The setup of my question is as follows:
I have a source sending a UDP Packet to my receiving computer
Receiving computer takes the UDP packet and receives it into unsigned char *message.
I can print the packet out byte-wise using
for(int i = 0; i < sizeof(message); i++) {
printf("0x%02 \n", message[i];
}
And this is where I am! Now I'd like to start parsing these bytes I recieved into the network as shorts, ints, longs, and strings.
I've written a series of functions like:
short unsignedShortToInt(char[] c) {
short i = 0;
i |= c[1] & 0xff;
i <<= 8;
i |= c[0] & 0xff;
return i;
}
to parse the bytes and shift them into ints, longs, and shorts. I can use sprintf() to create strings from byte arrays.
My question is -- what's the best way to get the substrings from my massive UDP packet? The packet is over 100 character in lengths, so I'd like an easy way to pass in message[0:6] or message[20:22] to these variation utility functions.
Possible options:
I can use strcpy() to create a temporary array for each function call, but that seems a bit messy.
I can turn the entire packet into a string and use std::string::substr. This seems nice, but I'm concerned that converting the unsigned chars into signed chars (part of the string conversion process) might cause some errors (maybe this concern is unwarranted?).
Maybe another way?
So I ask you, stackoverflow, to recommend a clean, concise way to do this task!
thanks!
Why not use proper serialization ?
i.e. MsgPack
You'll need a scheme how to differentiate messages. You could for example make them self-describing, something like:
struct my_message {
string protocol;
string data;
};
and dispatch decoding based on the protocol.
You'll most probably be better off using a tested serialization library than finding out that your system is vulnerable to buffer overflow attacks and malfunction.
I think you have two problems to solve here. First you need to make sure the integer data are properly aligned in memory after extracting them from the character buffer. next you need to ensure the correct byte order of the integer data after their extraction.
The alignment problem can be solved with a union containing the integral data type super-imposed upon a character array of the correct size. The network byte order problem can be solved using the standard ntohs() and ntohl() functions. This will only work if the sending software also used the standard byte-order produced by the inverse of these functions.
See: http://www.beej.us/guide/bgnet/output/html/multipage/htonsman.html
Here are a couple of UNTESTED functions you may find useful. I think they should just about do what you are after.
#include <netinet/in.h>
/**
* General routing to extract aligned integral types
* from the UDP packet.
*
* #param data Pointer into the UDP packet data
* #param type Integral type to extract
*
* #return data pointer advanced to next position after extracted integral.
*/
template<typename Type>
unsigned char const* extract(unsigned char const* data, Type& type)
{
// This union will ensure the integral data type is correctly aligned
union tx_t
{
unsigned char cdata[sizeof(Type)];
Type tdata;
} tx;
for(size_t i(0); i < sizeof(Type); ++i)
tx.cdata[i] = data[i];
type = tx.tdata;
return data + sizeof(Type);
}
/**
* If strings are null terminated in the buffer then this could be used to extract them.
*
* #param data Pointer into the UDP packet data
* #param s std::string type to extract
*
* #return data pointer advanced to next position after extracted std::string.
*/
unsigned char const* extract(unsigned char const* data, std::string& s)
{
s.assign((char const*)data, std::strlen((char const*)data));
return data + s.size();
}
/**
* Function to parse entire UDP packet
*
* #param data The entire UDP packet data
*/
void read_data(unsigned char const* const data)
{
uint16_t i1;
std::string s1;
uint32_t i2;
std::string s2;
unsigned char const* p = data;
p = extract(p, i1); // p contains next position to read
i1 = ntohs(i1);
p = extract(p, s1);
p = extract(p, i2);
i2 = ntohl(i2);
p = extract(p, s2);
}
Hope that helps.
EDIT:
I have edited the example to include strings. It very much depends on how the strings are stored in the stream. This example assumes the strings are null-terminated c-strings.
EDIT2:
Whoopse, changed code to accept unsigned chars as per question.
If the array is only 100 characters in length just create a char buffer[100] and a queue of them so you don't miss processing any of the messages.
Next you could just index that buffer as you described and if you know the struct of the message then you know the index points.
next you can union the types i.e
union myType{
char buf[4];
int x;
}
giving you the value as an int from a char if thats what you need
I used struct.pack in Python to transform a data into serialized byte stream.
>>> import struct
>>> struct.pack('i', 1234)
'\xd2\x04\x00\x00'
What is the equivalence in C++?
You'll probably be better off in the long run using a third party library (e.g. Google Protocol Buffers), but if you insist on rolling your own, the C++ version of your example might be something like this:
#include <stdint.h>
#include <string.h>
int32_t myValueToPack = 1234; // or whatever
uint8_t myByteArray[sizeof(myValueToPack)];
int32_t bigEndianValue = htonl(myValueToPack); // convert the value to big-endian for cross-platform compatibility
memcpy(&myByteArray[0], &bigEndianValue, sizeof(bigEndianValue));
// At this point, myByteArray contains the "packed" data in network-endian (aka big-endian) format
The corresponding 'unpack' code would look like this:
// Assume at this point we have the packed array myByteArray, from before
int32_t bigEndianValue;
memcpy(&bigEndianValue, &myByteArray[0], sizeof(bigEndianValue));
int32_t theUnpackedValue = ntohl(bigEndianValue);
In real life you'd probably be packing more than one value, which is easy enough to do (by making the array size larger and calling htonl() and memcpy() in a loop -- don't forget to increase memcpy()'s first argument as you go, so that your second value doesn't overwrite the first value's location in the array, and so on).
You'd also probably want to pack (aka serialize) different data types as well. uint8_t's (aka chars) and booleans are simple enough as no endian-handling is necesary for them -- you can just copy each of them into the array verbatim as a single byte. uint16_t's you can convert to big-endian via htons(), and convert back to native-endian via ntohs(). Floating point values are a bit tricky, since there is no built-in htonf(), but you can roll your own that will work on IEEE754-compliant machines:
uint32_t htonf(float f)
{
uint32_t x;
memcpy(&x, &f, sizeof(float));
return htonl(x);
}
.... and the corresponding ntohf() to unpack them:
float ntohf(uint32_t nf)
{
float x;
nf = ntohl(nf);
memcpy(&x, &nf, sizeof(float));
return x;
}
Lastly for strings you can just add the bytes of the string to the buffer (including the NUL terminator) via memcpy:
const char * s = "hello";
int slen = strlen(s);
memcpy(myByteArray, s, slen+1); // +1 for the NUL byte
There isn't one. C++ doesn't have built-in serialization.
You would have to write individual objects to a byte array/vector, and being careful about endianness (if you want your code to be portable).
https://github.com/karkason/cppystruct
#include "cppystruct.h"
// icmp_header can be any type that supports std::size and std::data and holds bytes
auto [type, code, checksum, p_id, sequence] = pystruct::unpack(PY_STRING("bbHHh"), icmp_header);
int leet = 1337;
auto runtimePacked = pystruct::pack(PY_STRING(">2i10s"), leet, 20, "String!");
// runtimePacked is an std::array filled with "\x00\x00\x059\x00\x00\x00\x10String!\x00\x00\x00"
// The format is "compiled" and has zero overhead in runtime
constexpr auto packed = pystruct::pack(PY_STRING("<2i10s"), 10, 20, "String!");
// packed is an std::array filled with "\x00\x01\x00\x00\x10\x00\x00\x00String!\x00\x00\x00"
You could check out Boost.Serialization, but I doubt you can get it to use the same format as Python's pack.
I was also looking for the same thing. Luckily I found https://github.com/mpapierski/struct
with a few additions you can add missing types into struct.hpp, I think it's the best so far.
To use it, just define you params like this
DEFINE_STRUCT(test,
((2, TYPE_UNSIGNED_INT))
((20, TYPE_CHAR))
((20, TYPE_CHAR))
)
The just call this function which will be generated at compilation
pack(unsigned int p1, unsigned int p2, const char * p3, const char * p4)
The number and type of parameters will depend on what you defined above.
The return type is a char* which contains your packed data.
There is also another unpack() function which you can use to read the buffer
You can use union to get different view into the same memory.
For example:
union Pack{
int i;
char c[sizeof(int)];
};
Pack p = {};
p.i = 1234;
std::string packed(p.c, sizeof(int)); // "\xd2\x04\x00\0"
As mentioned in the other answers, you have to notice the endianness.
I have a structure:
struct {
Header header;
uint32_t var1;
uint32_t var2;
char var3;
char var4[4];
};
You get the hint. The thing is that I am receiving byte arrays over the network, and I first have to parse the Header first. So I first parse the header first, and then I have to parse the rest of the structure.
I tried,
void* V = data; // which is sizeof(uint32_t) * 2 + sizeof(char) * 5
and then try to parse it like (V), V+sizeof(uint32_t) ... etc. etc.
but it gave compiler errors. How do I parse the rest of this struct over the network?
The fundamental unit of data in C++ is char. It is the smallest type that can be addressed, and it has size one by definition. Moreover, the language rules specifically allow all data to be viewed as a sequence of chars. All I/O happens in terms of sequences (or streams) of chars.
Therefore, your raw data buffer should be a char array.
(On the other hand, a void * has very specific and limited use in C++; it's main purpose is to designate an object's address in memory. For example, the result of operator new() is a void *.)
I have question about interpreting strings as packed binary data in C++. In python, I can use struct module. Is there a module or a way in C++ to interpret strings as packed binary data without embedding Python?
As already mentioned, it is better to consider this an array of bytes (chars, or unsigned chars), possibly held in a std::vector, rather than a string. A string is null terminated, so what happens if a byte of the binary data had the value zero?
You can either cast a pointer within the array to a pointer to your struct, or copy the data over a struct:
#include <memory>
#pragma pack ( push )
#pragma pack( 1 );
struct myData
{
int data1;
int data2;
// and whatever
};
#pragma pack ( pop )
char* dataStream = GetTheStreamSomehow();
//cast the whole array
myData* ptr = reinterpret_cast<myData*>( dataStream );
//cast from a known position within the array
myData* ptr2 = reinterpret_cast<myData*>( &(dataStream[index]) );
//copy the array into a struct
myData data;
memcpy( &data, dataStream, sizeof(myData) );
If you were to have the data stream in a vector, the [] operator would still work. The pragma pack declarations ensure the struct is single byte aligned - researching this is left as an exercise for the reader. :-)
Basically, you don't need to interpret anything. In C++, strings are
packed binary data; you can interpret them as text, but you're not
required to. Just be aware that the underlying type of a string, in
C++, is char, which can be either signed (range [-128,127] on all
machines I've heard of) or unsigned (usually [0,255], but I'm aware of
machines where it is [0,511]).
To pass the raw data in a string to a C program, use
std::string::data() and std::string::size(). Otherwise, you can
access it using iterators or indexation much as you would with
std::vector<char> (which may express the intent better).
A string in C++ has a method called c_str ( http://www.cplusplus.com/reference/string/string/c_str/ ).
c_str returns the relevant binary data in a string in form of an array of characters. You can cast these chars to anything you wish and read them as an array of numbers.
Eventhough it might be closer to pickling in python, boost serialization may be closest to what you want to achieve.
Otherwise you might want to do it by hand. It is not that hard to make reader/writer classes to convert primitives/classes to packed binary format. I would do it by shifting bytes to avoid host endianess issues.