I have this struct:
typedef struct pkt {
unsigned int pktid;
unsigned int pkt_leng;
unsigned int strleng;
char* str;
};
in my code I'm doing this:
mystring = someFunctionWhoReturnString();
pkt* mypkt = (pkt*) malloc(sizeof(pkt));
mypkt -> pkt = htonl(C_MYPKT);
mypkt -> pkt_leng = htonl(sizeof(pkt));
mpykt -> strleng = htonl(mystring.lengh());
mypkt -> str = (char*)malloc(mystring.length());
strcpy(mypkt -> str, mystring.c_str(), mystring.length());
after this, if i check what is stored on mypkt->str, the string its there.
but when i receive the pkt, i got just some garbage (the size is fine) where suppose to get the char* (the rest of the data in the packet arrive ok).
There is some smart way to accomplish this task without use a char[] with static size?
Im working on VC++ 2010.
Realize that sizeof(pkt) == 12, the size of three integers and a pointer. It doesn't include the length of the data the pointer points to. What you are sending over is three integers and the address of the string on the sending machine - an address that's completely useless to the receiving machine, of course.
Instead, you need to prepare a flat buffer that would have three integers immediately followed by character data.
Related
I made a function that serializes settings and returns a char* containing serialized data.
First i'm packing all the values into a StaticJsonDocument, then determining size of the output string using measureJson, then allocating space for the output char out[strsize] and then serializing data into space allocated serializeJson(doc,out,strsize)
The problem is that output string remains empty for unknown reason.
Things i checked:
Json document is constructed properly and actually contains configurations settings
measureJson() function properly returns the size of output and space is being allocated, strsize is not 0
Code:
char* configSerialize(bool msgpack){
StaticJsonDocument<settsize> doc;
JsonArray ipk = doc.createNestedArray("ip");
JsonArray gateipk = doc.createNestedArray("gateip");
JsonArray dnsk = doc.createNestedArray("dns");
JsonArray mack = doc.createNestedArray("mac");
unsigned char i;
for(i=0;i<4;i++){
ipk.add(ip[i]);
gateipk.add(gateip[i]);
dnsk.add(dns[i]);
}
for(i=0;i<6;i++){
mack.add(mac[i]);
}
doc["subnet"] = subnet;
doc["dhcp"] = DHCP;
doc["alertbuzz"] = alertbuzz;
const size_t strsize = msgpack ? measureMsgPack(doc) : measureJson(doc);
char out[strsize];
if(msgpack) serializeMsgPack(doc,out,strsize);
else serializeJson(doc,out,strsize);
return out;
}
char out[strsize];
This is a local variable/array inside your configSerialize() function and is invalid once you return from that function.
One way would be to use new and delete to allocate/deallocate space on the heap, but I would not recommend that on Arduino.
Another way would be to use char out[FIXED_SIZE]; outside of your function - i.e. as a global variable.
Also, if you're planning to use out as a string pointer, you'll need to add a zero byte at the end (and allocate space for that extra byte).
I have a long array of char (coming from a raster file via GDAL), all composed of 0 and 1. To compact the data, I want to convert it to an array of bits (thus dividing the size by 8), 4 bytes at a time, writing the result to a different file. This is what I have come up with by now:
uint32_t bytes2bits(char b[33]) {
b[32] = 0;
return strtoul(b,0,2);
}
const char data[36] = "00000000000000000000000010000000101"; // 101 is to be ignored
char word[33];
strncpy(word,data,32);
uint32_t byte = bytes2bits(word);
printf("Data: %d\n",byte); // 128
The code is working, and the result is going to be written in a separate file. What I'd like to know is: can I do that without copying the characters to a new array?
EDIT: I'm using a const variable here just to make a minimal, reproducible example. In my program it's a char *, which is continually changing value inside a loop.
Yes, you can, as long as you can modify the source string (in your example code you can't because it is a constant, but I assume in reality you have the string in writable memory):
uint32_t bytes2bits(const char* b) {
return strtoul(b,0,2);
}
void compress (char* data) {
// You would need to make sure that the `data` argument always has
// at least 33 characters in length (the null terminator at the end
// of the original string counts)
char temp = data[32];
data[32] = 0;
uint32_t byte = bytes2bits(data);
data[32] = temp;
printf("Data: %d\n",byte); // 128
}
In this example by using char* as a buffer to store that long data there is not necessary to copy all parts into a temporary buffer to convert it to a long.
Just use a variable to step through the buffer by each 32 byte length period, but after the 32th byte there needs the 0 termination byte.
So your code would look like:
uint32_t bytes2bits(const char* b) {
return strtoul(b,0,2);
}
void compress (char* data) {
int dataLen = strlen(data);
int periodLen = 32;
char* periodStr;
char tmp;
int periodPos = periodLen+1;
uint32_t byte;
periodStr = data[0];
while(periodPos < dataLen)
{
tmp = data[periodPos];
data[periodPos] = 0;
byte = bytes2bits(periodStr);
printf("Data: %d\n",byte); // 128
data[periodPos] = tmp;
periodStr = data[periodPos];
periodPos += periodLen;
}
if(periodPos - periodLen <= dataLen)
{
byte = bytes2bits(periodStr);
printf("Data: %d\n",byte); // 128
}
}
Please than be careful to the last period, which could be smaller than 32 bytes.
const char data[36]
You are in violation of your contract with the compiler if you declare something as const and then modify it.
Generally speaking, the compiler won't let you modify it...so to even try to do so with a const declaration you'd have to cast it (but don't)
char *sneaky_ptr = (char*)data;
sneaky_ptr[0] = 'U'; /* the U is for "undefined behavior" */
See: Can we change the value of an object defined with const through pointers?
So if you wanted to do this, you'd have to be sure the data was legitimately non-const.
The right way to do this in modern C++ is by using std::string to hold your string and std::string_view to process parts of that string without copying it.
You can using string_view with that char array you have though. It's common to use it to modernize the classical null-terminated string const char*.
I am having a lot of trouble with GRPC when using byte array. This is by .proto
message myType {
int32 format = 1;
bytes data = 2;
}
I am using CPP for Server implementation and Java for Client. Using ByteString in Java is a breeze but cannot deserialize in CPP (byte[] being changed from what was being sent from Java).
buffer is a byte[] byte buffer[<large_size>] And I'm converting the byte array (it's an image) into a smaller byte array, and it's crashing when trying to convert the byte[] received from grpc. The conversion function in CPP is good as I used it with the same image before using GRPC
This is the deserialization code for CPP. Here "req" is a myType object, and buffer is a byte[]
myFormat = req->format();
dataLen = req->data().length();
memcpy(buffer, req->data().c_str(), dataLen);
From what I understand, req->data() is in cpp std::string format
On the client side, you should pass both the parameter and its length.
parameter.set_framemat(mat, 12);
Do check if the length of the array at the server side is not zero. Note that bytes is char array and grpc is de marshalling it as string. So if say the array is filled with null characters the data length comes as zero.
I was trying a similar use case
Proto file
message Parameters {
bytes framemat = 16;
};
Client Snippet
const char mat[12] = {0}
parameter.set_framemat(mat);
stream->Write(parameter);
Server Snippet
std::thread reader([&]() {
::nokia::nas::Parameters request;
while (stream->Read(&request))
{
//get the params
grpc::string mat = request.framemat();
logger->info(" Length of Bytes= {} , mat.length()}
Output was zero!
So I changed the client input to check with some char strings and sure enough the data length was coming as 4 at the server side; because this time it was a valid string and string length matched.
const char mat[12] = "sdsd";
Better way is to specify in the proto the data as well as the data length
message StreamBytes {
bytes data_bytes =1;
int32 data_length = 2;
};
I have to copy the following structure to a char[] buffer.
struct AMG_ANGLES {
unsigned char bIsEnCrypted;
unsigned char bIsError;
unsigned short usErrorFlag;
unsigned char byteNumDABs;
unsigned short usBagId;
unsigned short usKvMa;
unsigned char byteDataType;
};
AMG_ANGLES struct_data;
struct_data.bIsEnCrypted = 1;
struct_data.bIsError = 1;
struct_data.usErrorFlag = 2;
struct_data.byteNumDABs = 1;
struct_data.usBagId =10;
struct_data.usKvMa=20;
struct_data.byteDataType = 1;
// here I am coping structure to a char buffer
char sendbuf[sizeof(struct_data)] = "";
memcpy(sendbuf,(char*)&struct_data, sizeof(struct_data));
on copy the buffer having first two unsigned char data and short (1,1,2) and size is only 3 bytes. reaming data was not copying.
Please help where i am doing wrong.
I tried following way also
memcpy(sendbuf+0, &struct_data.bIsEnCrypted, sizeof(struct_data.bIsEnCrypted));
memcpy(sendbuf+1, &struct_data.bIsError, sizeof(struct_data.bIsError));
memcpy(sendbuf+2, &struct_data.usErrorFlag, sizeof(struct_data.usErrorFlag));
memcpy(sendbuf+4, &struct_data.byteNumDABs, sizeof(struct_data.byteNumDABs));
memcpy(sendbuf+6, &struct_data.usBagId, sizeof(struct_data.usBagId));
memcpy(sendbuf+8, &struct_data.usKvMa, sizeof(struct_data.usKvMa));
memcpy(sendbuf+10, &struct_data.byteDataType, sizeof(struct_data.byteDataType));
same result I am getting.
Your code is fine; your approach to determine whether the contents of the buffer are correct is flawed.
You have not told us how you have determined that the contents of the buffer are wrong, but from your description I suspect that you did something like printf( "%s\n", sendbuf ). Well, that won't work, because your buffer does not really contain characters, it contains binary data.
Specifically, your short usErrorFlag is two bytes long, and since the value you store in it is 2, this means that it will be stored in sendbuf in two consecutive bytes, one byte will have the value of 0x02 and the next byte will have the value of 0x00. (Assuming, from hints in your description, that your hardware is "Little Endian".) So, when you try to view the contents of your sendbuf as a string, printf() will stop printing as soon as it encounters the 0x00 byte.
So, your code is correct. Proceed with sending your sendbuf to your UDP socket.
If I read "sendbuf" I immediately assume that you are sending data from one computer to another. These computers will have different compilers, the compilers will for example order their bytes in a different order. memcpy isn't going to work on all compilers.
I suggest you find where the contents of sendbuf is documented, and assign the individual bytes accordingly. For example
sendbuf [0] = struct_data.bIsEncrypted;
sendbuf [1] = struct_data.bIsError;
sendbuf [2] = struct_data.uIsErrorFlag >> 8;
sendbuf [3] = struct_data.uIsErrorFlag & 0xff;
This makes your code independent of byte ordering, independent of struct padding, independent of reordering of items once you are not using a POD, and so on. In your case I would bet money that there is at least padding between byteNumDABs and usBagId, and at the end.
(Bytes 2 and 3 might be exactly the other way round, that's why you find a spec for that data structure).
I'm currently working on a small C++ project where I use a client-server model someone else built. Data gets sent over the network and in my opinion it's in the wrong order. However, that's not something I can change.
Example data stream (simplified):
0x20 0x00 (C++: short with value 32)
0x10 0x35 (C++: short with value 13584)
0x61 0x62 0x63 0x00 (char*: abc)
0x01 (bool: true)
0x00 (bool: false)
I can represent this specific stream as :
struct test {
short sh1;
short sh2;
char abc[4];
bool bool1;
bool bool2;
}
And I can typecast it with test *t = (test*)stream; However, the char* has a variable length. It is, however, always null terminated.
I understand that there's no way of actually casting the stream to a struct, but I was wondering whether there would be a better way than struct test() { test(char* data) { ... }} (convert it via the constructor)
This is called Marshalling or serialization.
What you must do is read the stream one byte at a time (or put all in a buffer and read from that), and as soon as you have enough data for a member in the structure you fill it in.
When it comes to the string, you simply read until you hit the terminating zero, and then allocate memory and copy the string to that buffer and assign it to a pointer in the struct.
Reading strings this way is simplest and most effective if you have of the message in a buffer already, because then you don't need a temporary buffer for the string.
Remember though, that with this scheme you have to manually free the memory containing the string when you are done with the structure.
Just add a member function that takes in the character buffer(function input parameter char *) and populates the test structure by parsing it.
This makes it more clear and readable as well.
If you provide a implicit conversion constructor then you create a menace which will do the conversion when you least expect it.
When reading variable length data from a sequence of bytes,
you shouldn't fit everything into a single structure or variable.
Pointers are also used to store this variable length.
The following suggestion, is not tested:
// data is stored in memory,
// in a different way,
// NOT as sequence of bytes,
// as provided
struct data {
short sh1;
short sh2;
int abclength;
// a pointer, maybe variable in memory !!!
char* abc;
bool bool1;
bool bool2;
};
// reads a single byte
bool readByte(byte* MyByteBuffer)
{
// your reading code goes here,
// character by character, from stream,
// file, pipe, whatever.
// The result should be true if not error,
// false if cannot rea anymore
}
// used for reading several variables,
// with different sizes in bytes
int readBuffer(byte* Buffer, int BufferSize)
{
int RealCount = 0;
byte* p = Buffer;
while (readByte(p) && RealCount <= BufferSize)
{
RealCount++
p++;
}
return RealCount;
}
void read()
{
// real data here:
data Mydata;
byte MyByte = 0;
// long enough, used to read temporally, the variable string
char temp[64000];
// fill buffer for string with null values
memset(temp, '\0', 64000);
int RealCount = 0;
// try read "sh1" field
RealCount = (readBuffer(&(MyData.sh1), sizeof(short)));
if (RealCount == sizeof(short))
{
// try read "sh2" field
RealCount = readBuffer(&(MyData.sh2), sizeof(short));
if (RealCount == sizeof(short))
{
RealCount = readBuffer(temp, 64000);
if (RealCount > 0)
{
// store real bytes count
MyData.abclength = RealCount;
// allocate dynamic memory block for variable length data
MyData.abc = malloc(RealCount);
// copy data from temporal buffer into data structure plus pointer
// arrays in "plain c" or "c++" doesn't require the "&" operator for address:
memcpy(MyData.abc, temp, RealCount);
// comented should be read as:
//memcpy(&MyData.abc, &temp, RealCount);
// continue with rest of data
RealCount = readBuffer(&(MyData.bool1), sizeof(bool));
if (RealCount > 0)
{
// continue with rest of data
RealCount = readBuffer(&(MyData.bool2), sizeof(bool));
}
}
}
}
} // void read()
Cheers.