I'm trying to create a C# wrapper for a C++ code that reads data from an HID. The code I have been given is pretty straight forward but not complete. Data received from the device is read into the buffer as below:
pTmpBuf = (U8 *)calloc( InputReportByteLength, sizeof(U8));
if (ReadFile( hDevice, pTmpBuf, InputReportByteLength, &nRead, NULL))
{
memcpy(`pAppBuffer`, pTmpBuf + 1, nRead-1);
}
I want to parse the data in the pAppBuffer into the struct that is defined as follows:
struct BAYER_CONTOUR_REPORT
{
unsigned char reportID; // HID report ID
unsigned char checkSum; // checksum for hostID + deviceID + data
unsigned char hostID // host ID assigned by communications manager
unsigned char deviceID; // device ID assigned by communications manager
unsigned char length; // length of data in buffer
unsigned char data[60]; // data send with message
};
How can this be done? Any help or pointers is appreciated.
Can I simply parse the data by casting struct object onto the buffer?
You can do the memcpy to the struct with the incoming buffer, provided you're sure the the incoming buffer or contents are aligned to structure definition.
for example
struct abc {
char a;
char b;
char c;
char d[2];
};
int main() {
char arr[5] = { 'a', 'b', 'c', 'd', 'e' };
struct abc sa;
memcpy(&sa, arr, 5);
return 0;
}
Here arr is incoming buffer, and with memcpy all the contents are copied appropriately.
Similarly, in your code you can do the following
struct BAYER_CONTOUR_REPORT bcr;
memcpy(&bcr, pAppBuffer, sizeof(struct BAYER_CONTOUR_REPORT))
Again, please mind the caveats that you need to be absolutely sure that size of struct struct BAYER_CONTOUR_REPORT and pAppBuffer is exactly same and the information is aligned to your structure
Related
I'm trying to write a program which will query a URL using curl and retrieve a string of bytes. The returned data than needs to be interpreted as various data types; an int followed by a sequence structures.
The curl write back function must have a prototype of:
size_t write_callback(char *ptr, size_t size, size_t nmemb, void *userdata);
I've seen various examples where the returned data is stored in a buffer either as characters directly in memory or as a string object.
If I have a character array, then I know that I can interpret a portion of it as a structure with code like this:
struct mystruct {
//define struct
};
char *buffer;
//push some data into the buffer
char *read_position;
read_position = buffer + 5;
test = (mystruct *)buffer;
I have two related questions. Firstly, is there a better way of using curl to retrieve binary data and pushing it into structures, rather than reading it directly into memory as characters. Secondly if reading into memory as a character buffer is the way to go, is my code above a sensible way to interpret the chunks of memory as different data types?
Things you need to consider when interpreting raw structures, especially over network:
The size of your data types;
The endianness of your data types;
Struct padding.
You should only use data types in your structure that are the correct size regardless of what compiler is used. That means for integers, you should use types from <cstdint>.
As for the endianness, you need to know if the data will arrive as big-endian or little-endian. I like to be explicit about it:
template< class T >
const char * ReadLittleEndian32( const char *buf, T & val )
{
static_assert( sizeof(T) == 4 );
val = T(buf[0]) | T(buf[1]) << 8 | T(buf[2]) << 16 | T(buf[3]) << 24;
return buf + sizeof(T);
}
template< class T >
const char * ReadBigEndian32( const char *buf, T & val )
{
static_assert( sizeof(T) == 4 );
val = T(buf[0]) << 24 | T(buf[1]) << 16 | T(buf[2]) << 8 | T(buf[3]);
return buf + sizeof(T);
}
//etc...
Finally, dealing with potential padding differences... I've already been naturally tending towards a 'deserialise' approach where each value is read and translated explicitly. The structure is no different:
struct Foo
{
uint16_t a;
int16_t b;
int32_t c;
const char * Read( const char * buf );
};
const char * Foo::Read( const char * buf )
{
buf = ReadLittleEndian16( buf, a );
buf = ReadLittleEndian16( buf, b );
buf = ReadLittleEndian32( buf, c );
return buf;
}
Notice the templating handles sign and other things in the data type, so that all we care about in the end is size. Also remember that data types such as float and double already have inherent endianness and should not be translated -- they can be read verbatim:
const char * ReadDouble( const char * buf, double & val )
{
val = *(double*)buf;
return buf + sizeof(double);
}
I am intercepting some packets, and then put them into an structure.
#pragma pack(push, 1)
struct PacketHeader {
short Size;
short Checksum;
short Index;
};
#pragma pack(pop)
I have a packet with PacketHeader and some other bytes that fill this structure:
struct STRUCT_SVC_ROOM_CREATE {
PacketHeader Header;
unsigned char TitleLength; // 1 byte
char* RoomTitle[23];
short Unknow;
short Unknow2;
short Password1;
short Password2;
char LastByte;
};
In the above struct, TitleLength is one byte, that in decimal can 0x17 (23) or any number. This number the numbers of chars contained in RoomTitle.
I need to set size of RoomTitle accortng to TitleLenght byte (as decimal number).
How could I modify the struct to handle the text size in the right location inside the struct?
You should do something like follows, to parse the RoomTitle from the packet received at your socket:
struct STRUCT_SVC_ROOM_CREATE {
PacketHeader Header; // Header length is sizeof(PacketHeader)
unsigned char TitleLength; // 1 byte
char RoomTitle[255]; // I suspect you don't have 23 `RoomTitle[23];` char*
// pointers at this point, but just a char* maximally
// sized as the maximum number that TitleLength can hold
// (which is 255).
short Unknow; // Unknow length is sizeof(short)
short Unknow2; // ... ditto aso.
short Password1;
short Password2;
char LastByte;
};
As I pointed out in the code comments above
Read the PacketHeader (take care of Size and CRC endianess!)
Read the payload data according to PacketHeader::Size from the packet into another buffer. (Consider to check the CRC)
Read the TitleLength and RoomTitle from the payload data accordingly. Take care, if you want to handle the RoomTitle data as a c-style string, it's actually terminated with '\0'. Also use the TitleLength information when copying elsewhere.
Read the data with well known size coming after (take care of endianess again)
Some pseudo code (not tested):
int recv_ROOM_CREATE_packet(int sockfd, STRUCT_SVC_ROOM_CREATE* packet) {
read(sockfd,&(packet->Header),sizeof(PacketHeader));
read(sockfd,&(packet->TitleLength),sizeof(unsigned char));
read(sockfd,packet->RoomTitle,packet->TitleLength);
// ensure that packet->RoomTitle is a correctly terminated c-style string
if(packet->TitleLength < 255) {
packet->RoomTitle[packet->TitleLength + 1] = `\0`;
}
else {
packet->RoomTitle[254] = `\0`;
}
// aso ...
}
I want to pass a character buffer to a proc entry (say /proc/my_file) from my program. This character buffer contains my structure's elements which is of the form:
struct my_table { char src_ip[4]; char dest_ip[4]; int out_flag; }my_t;
I assign the elements of my_table and copy its contents to an unsigned char buffer as follows:
memcpy(buffer, &my_t, sizeof(struct my_table));
Then I write the buffer's contents into a proc entry created by me (by the name my_file) as:
write(fd, buffer, sizeof(buffer));
where fd is the file descriptor returned by open() after opening the /proc/my_file with O_WRONLY | O_APPEND flags.
What I couldn't understand is that I can only see the first character string i.e my_t.src_ip in this case to be written to /proc/my_file (did a
cat /proc/my_file
to check the content written) and subsequently I observed that the write() operation to /proc/my_file ends as soon as it encounters a null character in the buffer content.
Can I know why does this happen and how to solve this problem of writing a structure's contents into a /proc entry?
Edit: SSCCE for my question:
The structure:
struct my_iptable {
char protocol[5]; // to check whether the protocol mentioned is tcp/udp/icmp
char src_ip[16]; // source ip address
char dest_ip[16]; // destination ip address
char src_net_mask[16]; // source net mask
char dest_net_mask[16]; // destination net mask
int src_port; // source port number
int dest_port; // destination port number
char action[8]; // either block or unblock
int delete_rule; // gets the rule number to be deleted
int print_flag; // is set if we are asked to print the rules that have been set
int out_flag; // is set if the packet is outbound, else set to 0;
};
The assignment of my_ipt to null:
struct my_iptable my_ipt;
memset(&my_ipt, '\0', sizeof(struct my_iptable));
I have assigned my_ipt's fields properly.
The copying to buffer and writing to proc part:
unsigned char write_buf[sizeof(struct my_iptable)];
memcpy(write_buf, &my_ipt, sizeof(struct my_iptable));
int proc_fp = open("/proc/minifw", O_WRONLY | O_APPEND);
if(proc_fp < 0) {
printf("Couldn't open /proc/minifw for writing\n");
exit(EXIT_FAILURE);}
if(write(proc_fp, write_buf, sizeof(struct my_iptable)) == -1) {
printf("There was an error writing to minifw buffer\n");
exit(EXIT_FAILURE);
}
I hope this gives you appropriate info on what I want to understand.
Thanks!
use sizeof(struct my_table) instead
write(fd, buffer, sizeof(struct my_table));
If your buffer is defined as pointer:
struct my_table *buffer;
then the size of buffer will be equal to the size of pointer (4 for 32-bit systems and 8 for 64-bit systems) and not the real size of the struct my_table
I'm currently working on a small C++ project where I use a client-server model someone else built. Data gets sent over the network and in my opinion it's in the wrong order. However, that's not something I can change.
Example data stream (simplified):
0x20 0x00 (C++: short with value 32)
0x10 0x35 (C++: short with value 13584)
0x61 0x62 0x63 0x00 (char*: abc)
0x01 (bool: true)
0x00 (bool: false)
I can represent this specific stream as :
struct test {
short sh1;
short sh2;
char abc[4];
bool bool1;
bool bool2;
}
And I can typecast it with test *t = (test*)stream; However, the char* has a variable length. It is, however, always null terminated.
I understand that there's no way of actually casting the stream to a struct, but I was wondering whether there would be a better way than struct test() { test(char* data) { ... }} (convert it via the constructor)
This is called Marshalling or serialization.
What you must do is read the stream one byte at a time (or put all in a buffer and read from that), and as soon as you have enough data for a member in the structure you fill it in.
When it comes to the string, you simply read until you hit the terminating zero, and then allocate memory and copy the string to that buffer and assign it to a pointer in the struct.
Reading strings this way is simplest and most effective if you have of the message in a buffer already, because then you don't need a temporary buffer for the string.
Remember though, that with this scheme you have to manually free the memory containing the string when you are done with the structure.
Just add a member function that takes in the character buffer(function input parameter char *) and populates the test structure by parsing it.
This makes it more clear and readable as well.
If you provide a implicit conversion constructor then you create a menace which will do the conversion when you least expect it.
When reading variable length data from a sequence of bytes,
you shouldn't fit everything into a single structure or variable.
Pointers are also used to store this variable length.
The following suggestion, is not tested:
// data is stored in memory,
// in a different way,
// NOT as sequence of bytes,
// as provided
struct data {
short sh1;
short sh2;
int abclength;
// a pointer, maybe variable in memory !!!
char* abc;
bool bool1;
bool bool2;
};
// reads a single byte
bool readByte(byte* MyByteBuffer)
{
// your reading code goes here,
// character by character, from stream,
// file, pipe, whatever.
// The result should be true if not error,
// false if cannot rea anymore
}
// used for reading several variables,
// with different sizes in bytes
int readBuffer(byte* Buffer, int BufferSize)
{
int RealCount = 0;
byte* p = Buffer;
while (readByte(p) && RealCount <= BufferSize)
{
RealCount++
p++;
}
return RealCount;
}
void read()
{
// real data here:
data Mydata;
byte MyByte = 0;
// long enough, used to read temporally, the variable string
char temp[64000];
// fill buffer for string with null values
memset(temp, '\0', 64000);
int RealCount = 0;
// try read "sh1" field
RealCount = (readBuffer(&(MyData.sh1), sizeof(short)));
if (RealCount == sizeof(short))
{
// try read "sh2" field
RealCount = readBuffer(&(MyData.sh2), sizeof(short));
if (RealCount == sizeof(short))
{
RealCount = readBuffer(temp, 64000);
if (RealCount > 0)
{
// store real bytes count
MyData.abclength = RealCount;
// allocate dynamic memory block for variable length data
MyData.abc = malloc(RealCount);
// copy data from temporal buffer into data structure plus pointer
// arrays in "plain c" or "c++" doesn't require the "&" operator for address:
memcpy(MyData.abc, temp, RealCount);
// comented should be read as:
//memcpy(&MyData.abc, &temp, RealCount);
// continue with rest of data
RealCount = readBuffer(&(MyData.bool1), sizeof(bool));
if (RealCount > 0)
{
// continue with rest of data
RealCount = readBuffer(&(MyData.bool2), sizeof(bool));
}
}
}
}
} // void read()
Cheers.
I have a struct
struct Packet {
int senderId;
int sequenceNumber;
char data[MaxDataSize];
char* Serialize() {
char *message = new char[MaxMailSize];
message[0] = senderId;
message[1] = sequenceNumber;
for (unsigned i=0;i<MaxDataSize;i++)
message[i+2] = data[i];
return message;
}
void Deserialize(char *message) {
senderId = message[0];
sequenceNumber = message[1];
for (unsigned i=0;i<MaxDataSize;i++)
data[i] = message[i+2];
}
};
I need to convert this to a char* , maximum length MaxMailSize > MaxDataSize for sending over network and then deserialize it at the other end
I can't use tpl or any other library.
Is there any way to make this better I am not that comfortable with this, or is this the best we can do.
since this is to be sent over a network, i strongly advise you to convert those data into network byte order before transmitting, and back into host byte order when receiving. this is because the byte ordering is not the same everywhere, and once your bytes are not in the right order, it may become very difficult to reverse them (depending on the programming language used on the receiving side). byte ordering functions are defined along with sockets, and are named htons(), htonl(), ntohs() and ntohl(). (in those name: h means 'host' or your computer, n means 'network', s means 'short' or 16bit value, l means 'long' or 32 bit value).
then you are on your own with serialization, C and C++ have no automatic way to perform it. some softwares can generate code to do it for you, like the ASN.1 implementation asn1c, but they are difficult to use because they involve much more than just copying data over the network.
Depending if you have enough place or not... you might simply use the streams :)
std::string Serialize() {
std::ostringstream out;
char version = '1';
out << version << senderId << '|' << sequenceNumber << '|' << data;
return out.str();
}
void Deserialize(const std::string& iString)
{
std::istringstream in(iString);
char version = 0, check1 = 0, check2 = 0;
in >> version;
switch(version)
{
case '1':
senderId >> check1 >> sequenceNumber >> check2 >> data;
break;
default:
// Handle
}
// You can check here than 'check1' and 'check2' both equal to '|'
}
I readily admit it takes more place... or that it might.
Actually, on a 32 bits architecture an int usually cover 4 bytes (4 char). Serializing them using streams only take more than 4 'char' if the value is superior to 9999, which usually gives some room.
Also note that you should probably include some guards in your stream, just to check when you get it back that it's alright.
Versioning is probably a good idea, it does not cost much and allows for unplanned later development.
You can have a class reprensenting the object you use in your software with all the niceties and member func and whatever you need. Then you have a 'serialized' struct that's more of a description of what will end up on the network.
To ensure the compiler will do whatever you tell him to do, you need to instruct it to 'pack' the structure. The directive I used here is for gcc, see your compiler doc if you're not using gcc.
Then the serialize and deserialize routine just convert between the two, ensuring byte order and details like that.
#include <arpa/inet.h> /* ntohl htonl */
#include <string.h> /* memcpy */
class Packet {
int senderId;
int sequenceNumber;
char data[MaxDataSize];
public:
char* Serialize();
void Deserialize(char *message);
};
struct SerializedPacket {
int senderId;
int sequenceNumber;
char data[MaxDataSize];
} __attribute__((packed));
void* Packet::Serialize() {
struct SerializedPacket *s = new SerializedPacket();
s->senderId = htonl(this->senderId);
s->sequenceNumber = htonl(this->sequenceNumber);
memcpy(s->data, this->data, MaxDataSize);
return s;
}
void Packet::Deserialize(void *message) {
struct SerializedPacket *s = (struct SerializedPacket*)message;
this->senderId = ntohl(s->senderId);
this->sequenceNumber = ntohl(s->sequenceNumber);
memcpy(this->data, s->data, MaxDataSize);
}
int senderId;
int sequenceNumber;
...
char *message = new char[MaxMailSize];
message[0] = senderId;
message[1] = sequenceNumber;
You're overwriting values here. senderId and sequenceNumber are both ints and will take up more than sizeof(char) bytes on most architectures. Try something more like this:
char * message = new char[MaxMailSize];
int offset = 0;
memcpy(message + offset, &senderId, sizeof(senderId));
offset += sizeof(senderId);
memcpy(message + offset, &sequenceNumber, sizeof(sequenceNumber));
offset += sizeof(sequenceNumber);
memcpy(message + offset, data, MaxDataSize);
EDIT:
fixed code written in a stupor. Also, as noted in comment, any such packet is not portable due to endian differences.
To answer your question generally, C++ has no reflection mechanism, and so manual serialize and unserialize functions defined on a per-class basis is the best you can do. That being said, the serialization function you wrote will mangle your data. Here is a correct implementation:
char * message = new char[MaxMailSize];
int net_senderId = htonl(senderId);
int net_sequenceNumber = htonl(sequenceNumber);
memcpy(message, &net_senderId, sizeof(net_senderId));
memcpy(message + sizeof(net_senderId), &net_sequenceNumber, sizeof(net_sequenceNumber));
As mentioned in other posts, senderId and sequenceNumber are both of type int, which is likely to be larger than char, so these values will be truncated.
If that's acceptable, then the code is OK. If not, then you need to split them into their constituent bytes. Given that the protocol you are using will specifiy the byte order of multi-byte fields, the most portable, and least ambiguous, way of doing this is through shifting.
For example, let's say that senderId and sequenceNumber are both 2 bytes long, and the protocol requires that the higher byte goes first:
char* Serialize() {
char *message = new char[MaxMailSize];
message[0] = senderId >> 8;
message[1] = senderId;
message[2] = sequenceNumber >> 8;
message[3] = sequenceNumber;
memcpy(&message[4], data, MaxDataSize);
return message;
}
I'd also recommend replacing the for loop with memcpy (if available), as it's unlikely to be less efficient, and it makes the code shorter.
Finally, this all assumes that char is one byte long. If it isn't, then all the data will need to be masked, e.g.:
message[0] = (senderId >> 8) & 0xFF;
You can use Protocol Buffers for defining and serializing of structs and classes. This is what google uses internally, and has a very small transfer mechanism.
http://code.google.com/apis/protocolbuffers/