How do I deserialise a const byte * to a structure in cpp? - c++

I have a structure like this
struct foo {
string str1;
uint16_t int1
string str2;
uint32_t int2;
string str3;
};
strings str1, str2 , str3 are of fixed length of 12 bytes, 3 bytes,etc. left padded with spaces.
I have a function
void func(const byte* data, const size_t len) which is supposed to convert the byte * data to structure foo. len is length of data.What are the ways in which I can do this?
Again the data is const pointer of byte type and will not have null characters in between to distinguish different members.
Should I use character array instead of string for str1, str2, str3?

Easiest (but most errorprone) way is to just reinterpret_cast / std::memcpy if the strings have fixed length:
// no padding
#pragma pack(push, 1)
struct foo {
char str1[12];
uint16_t int1;
char str2[3];
uint32_t int2;
char str3[4];
};
#pragma pack(pop)
void func(const byte* data, const size_t len) {
assert(len == sizeof(foo));
// non owning
const foo* reinterpreted = reinterpret_cast<const foo*>(data);
// owning
foo reinterpreted_val = *reinterpret_cast<const foo*>(data);
foo copied;
memcpy(&copied, data, len);
}
Notes:
Make sure that you're allowed to use reinterpret_cast
https://en.cppreference.com/w/cpp/language/reinterpret_cast#Type_aliasing
if you'd try to use strlen or another string operation on any of the strings you most likely will get UB, since the strings are not null terminated.
Slightly better approach:
struct foo {
char str1[13];
uint16_t int1;
char str2[4];
uint32_t int2;
char str3[5];
};
void func(const char* data, const size_t len) {
foo f;
memcpy(f.str1, data, 12);
f.str1[12] = '\0';
data+=12;
memcpy(&f.int1, data, sizeof(uint16_t));
data+=sizeof(uint16_t);
memcpy(f.str2, data, 3);
f.str2[3] = '\0';
data+=3;
memcpy(&f.int2, data, sizeof(uint32_t));
data+=sizeof(uint32_t);
memcpy(f.str3, data, 4);
f.str3[4] = '\0';
data+=4;
}
Notes:
You could combine both approaches to get rid of the pointer arithmetic. That would also account for any padding in your struct you might have.

I think the easiest way to do this is to change the string inside the structure
to the type of char. Then you can easily copy the objects of this
structure according to its size.
you will have to somehow deal with the byte order on machines with different byte
order
struct foo {
char str1[12];
uint16_t int1;
char str2[3];
uint32_t int2;
char str3[5];
};
byte* Encode(foo* p, int Size) {
int FullSize = Size * sizeof(foo);
byte* Returner = new byte[FullSize];
memcpy_s(Returner, FullSize, p, FullSize);
return Returner;
}
foo * func(const byte* data, const size_t len) {
int ArrSize = len/sizeof(foo);
if (!ArrSize || (ArrSize* sizeof(foo)!= len))
return nullptr;
foo* Returner = new foo[ArrSize];
memcpy_s(Returner, len, data, len);
return Returner;
}
int main()
{
const size_t ArrSize = 3;
foo Test[ArrSize] = { {"Test1",1000,"TT",2000,"cccc"},{"Test2",1001,"YY",2001,"vvvv"},{"Test1",1002,"UU",2002,"bbbb"}};
foo* Test1 = nullptr;
byte* Data = Encode(Test, ArrSize);
Test1 = func(Data, ArrSize * sizeof(foo));
if (Test1 == nullptr) {
std::cout << "Error extracting data!" << std::endl;
delete [] Data;
return -1;
}
std::cout << Test1[0].str1 << " " << Test1[1].str1 << " " << Test1[2].str3 << std::endl;
delete [] Data;
delete[] Test1;
return 0;
}
output
Test1 Test2 bbbb

Related

C++ Union Array differs in 32/64 bits

My code:
union FIELD {
int n;
char c;
const char *s;
FIELD(){}
FIELD(int v){ n = v; }
FIELD(char v){ c = v; }
FIELD(const char* v){ s = v; }
};
struct SF {
const char* s0;
char s1;
int s2;
const char* s3;
};
int main() {
printf("sizeof(long) = %ld\n", sizeof(long));
printf("now is %d bit\n", sizeof(long) == 8?64:32);
FIELD arrField[] = {
FIELD("any 8 words 0 mixed"), FIELD('d'), FIELD(251356), FIELD("edcba")
};
SF* sf0 = (SF*)&arrField;
printf("sf0->s0 = %s, ", sf0->s0);
printf("sf0->s1 = %c, ", sf0->s1);
printf("sf0->s2 = %d, ", sf0->s2);
printf("sf0->s3 = %s\n", sf0->s3);
}
When I use the default 64-bit execution output:
I add the compilation parameters in CMakeLists.txt:
set_target_properties(untitled PROPERTIES COMPILE_FLAGS "-m32" LINK_FLAGS "-m32")
It will compile the 32-bit program, then run and output:
My question is, how can I make a 64-bit program have the same output behavior as a 32-bit program?
Apply alignas(FIELD) to every single member variable of SF.
Additionally you cannot rely on the size of long to tell 64 bit and 32 bit systems appart. Check the size of a pointer to do this. On some 64 bit systems long is 32 bit. This is the case for my system for example.
Furthermore %ld requires a long parameter, but the sizeof operator yields size_t which is unsigned in addition to not necesarily matching long in size. You need to add a cast there to be safe (or just go with std::cout which automatically chooses the correct conversion based on the second operand of the << operator).
union FIELD {
int n;
char c;
const char* s;
FIELD() {}
FIELD(int v) { n = v; }
FIELD(char v) { c = v; }
FIELD(const char* v) { s = v; }
};
struct SF {
alignas(FIELD) const char* s0;
alignas(FIELD) char s1;
alignas(FIELD) int s2;
alignas(FIELD) const char* s3;
};
int main() {
printf("sizeof(long) = %ld\n", static_cast<long>(sizeof(long)));
printf("now is %d bit\n", static_cast<int>(sizeof(void*)) * 8);
FIELD arrField[] = {
FIELD("any 8 words 0 mixed"), FIELD('d'), FIELD(251356), FIELD("edcba")
};
SF* sf0 = (SF*)&arrField;
printf("sf0->s0 = %s, ", sf0->s0);
printf("sf0->s1 = %c, ", sf0->s1);
printf("sf0->s2 = %d, ", sf0->s2);
printf("sf0->s3 = %s\n", sf0->s3);
}

How to allocate memory for array of structs?

I've a C++ library (really a wrapper for another C++ library) and I need pass some structs to my C application.
I don't know how allocate the memory dynamically.
I get a segmentation fault.
library.h
struct my_substruct {
unsigned char id ;
time_t date ;
char *info ;
};
typedef struct my_substruct My_substruct ;
struct my_struct {
char *description ;
unsigned char value ;
My_substruct *substruct ;
};
typedef my_struct My_struct ;
library.cpp
unsigned char getStructs(My_struct *structs)
{
vector <structCPPLibrary> structsCPPLibrary = getFromCPPLibrary();
unsigned char numStructs = structsCPPLibrary.size();
structs = (My_struct *)malloc(numStructs*sizeof(My_struct));
unsigned char indexStruct = 0;
for (auto s : structsCPPLibrary)
{
structs[indexStruct].description = (char *)malloc(s.description.size() + 1);
strcpy(structs[indexStruct].description, s.description.c_str()); // In 's' is a std::string
structs[indexStruct].value = s.value; // In 's' is an unsigned char too
unsigned char numSubstructs = s.substructs.size(); // In 's' is a vector of Substructs
structs[indexStruct].substruct = (My_substruct *)malloc(numSubstructs*sizeof(My_substruct));
unsigned char indexSubstruct = 0;
for (auto sub : s.substruct) {
structs[indexStruct].substruct[indexSubstruct].id = sub.id; // In 'sub' is an unsigned char too
structs[indexStruct].substruct[indexSubstruct].date = sub.date; // In 'sub' is a time_t too
structs[indexStruct].substruct[indexSubstruct].info = (char *)malloc(sub.info.size() + 1);
strcpy(structs[indexStruct].substruct[indexSubstruct].info, sub.info.c_str()); // In 'sub' is a std::string
indexSubstruct++;
}
indexStruct++;
}
return indexStruct;
}
main.c
void getStructFromWrapper(void)
{
My_struct *structs;
unsigned char numStruct = getStructs(structs);
show_content(structs);
}
Change
unsigned char getStructs(My_struct *structs) {
...
}
getStructs(structs);
To
unsigned char getStructs(My_struct **p_structs) {
// C function can't be pass by reference, so convert to a reference here
auto& struct = *p_structs;
...
}
...
getStructs(&structs);
Your problem is that your struct = ... line is not modifying the value of structs in getStructFromWrapper.

C++ object containing an array of char using unique_ptr

I am looking on a way to use unique_ptr to allocate a structure that contains an array of char with a number of bytes that set dynamically to support different types of message.
Assuming:
struct MyMessage
{
uint32_t id;
uint32_t data_size;
char data[4];
};
How can I convert send_message() below to use a smart pointer?
void send_message(void* data, const size_t data_size)
{
const auto message_size = sizeof(MyMessage) - 4 + data_size;
const auto msg = reinterpret_cast<MyMessage*>(new char[message_size]);
msg->id = 3;
msg->data_size = data_size;
memcpy(msg->data, data, data_size);
// Sending the message
// ...
delete[] msg;
}
My attempt to use smart point using the code below does not compile:
const auto message_size = sizeof(MyMessage) - 4 + data_size;
const auto msg = std::unique_ptr<MyMessage*>(new char[message_size]);
Below a complete working example:
#include <iostream>
#include <iterator>
#include <memory>
using namespace std;
struct MyMessage
{
uint32_t id;
uint32_t data_size;
char data[4];
};
void send_message(void* data, const size_t data_size)
{
const auto message_size = sizeof(MyMessage) - 4 + data_size;
const auto msg = reinterpret_cast<MyMessage*>(new char[message_size]);
if (msg == nullptr)
{
throw std::domain_error("Not enough memory to allocate space for the message to sent");
}
msg->id = 3;
msg->data_size = data_size;
memcpy(msg->data, data, data_size);
// Sending the message
// ...
delete[] msg;
}
struct MyData
{
int page_id;
char point_name[8];
};
void main()
{
try
{
MyData data{};
data.page_id = 7;
strcpy_s(data.point_name, sizeof(data.point_name), "ab332");
send_message(&data, sizeof(data));
}
catch (std::exception& e)
{
std::cout << "Error: " << e.what() << std::endl;
}
}
The data type that you pass to delete[] needs to match what new[] returns. In your example, you are new[]ing a char[] array, but are then delete[]ing a MyMessage object instead. That will not work.
The simple fix would be to change this line:
delete[] msg;
To this instead:
delete[] reinterpret_cast<char*>(msg);
However, You should use a smart pointer to manage the memory deletion for you. But, the pointer that you give to std::unique_ptr needs to match the template parameter that you specify. In your example, you are declaring a std::unique_ptr whose template parameter is MyMessage*, so the constructor is expecting a MyMessage**, but you are passing it a char* instead.
Try this instead:
// if this struct is being sent externally, consider
// setting its alignment to 1 byte, and setting the
// size of the data[] member to 1 instead of 4...
struct MyMessage
{
uint32_t id;
uint32_t data_size;
char data[4];
};
void send_message(void* data, const size_t data_size)
{
const auto message_size = offsetof(MyMessage, data) + data_size;
std::unique_ptr<char[]> buffer = std::make_unique<char[]>(message_size);
MyMessage *msg = reinterpret_cast<MyMessage*>(buffer.get());
msg->id = 3;
msg->data_size = data_size;
std::memcpy(msg->data, data, data_size);
// Sending the message
// ...
}
Or this:
using MyMessage_ptr = std::unique_ptr<MyMessage, void(*)(MyMessage*)>;
void send_message(void* data, const size_t data_size)
{
const auto message_size = offsetof(MyMessage, data) + data_size;
MyMessage_ptr msg(
reinterpret_cast<MyMessage*>(new char[message_size]),
[](MyMessage *m){ delete[] reinterpret_cast<char*>(m); }
);
msg->id = 3;
msg->data_size = data_size;
std::memcpy(msg->data, data, data_size);
// Sending the message
// ...
}
This should work, but it is still not clear if accessing msg->data out of bounds is legal (but at least it is not worst than in your original code):
const auto message_size = sizeof(MyMessage) - ( data_size < 4 ? 0 : data_size - 4 );
auto rawmsg = std::make_unique<char[]>( message_size );
auto msg = new (rawmsg.get()) MyMessage;

c++ compress with libzpaq to char* buffer

I have a std::vector<short> and would like to compress (and later decompress) with the libzpaq from https://github.com/zpaq/zpaq/ to something like char* buffer.
However I don't get the concept of this Reader and Writer class mentioned in the header file. How do I put my std::vector in to get a compressed buffer out?
Currently I have something like the following code.
#include <vector>
#include <string>
#include <stdio.h>
#include "libzpaq.h"
struct writer: public libzpaq::Writer {
void put(int c) {
}
};
struct reader: public libzpaq::Reader {
int get() {
}
};
void libzpaq::error(const char* msg) {
fprintf(stderr, "Oops: %s\n", msg);
exit(1);
}
int main() {
short a[] = {2,5,8,2,4,2,2,2,6,5,4,3,4,2,2};
std::vector<short> v(a, a+15);
char* buffer;
reader in;
writer out;
libzpaq::compress(&in, &out, "5");
}
And I wan't to compress the vector v into buffer. (And later decompress it again.)
But I don't understand the concept of the Reader and Writer struct/class.
The docu (http://mattmahoney.net/dc/libzpaq.3.html) also mentions the functions virtual int read(char* buf, int n) and virtual void write(const char* buf, int n) for the Reader and Writer. How can I cast a std::vector<short> to char* buf end get the length in n bytes of this buf?
Edit 1: I found a class StringBuffer in libzpaq.h line 1376. But something like
buffer = reinterpret_cast<char*> (&v[0]);
length = sizeof(short)*v.size();
libzpaq::StringBuffer inString, outString;
inString.read(buffer, length);
libzpaq::compress(&inString, &outString, "5");
std::cout << "size outstring: " << outString.size() << std::endl;
std::cout << "size instring: " << inString.size() << std::endl;
always gives me
size outstring: 0
size instring: 0
Even if I try it with a much larger vector v of some thousend random elements.
With Reader you provide byte by byte access to the data you want to compress. So with std::vector<short> it would look like this.
struct reader : public libzpaq::Reader {
reader(const std::vector<short>& v) :
m_v(v),
m_offset(0) {
}
int get() {
if (m_offset < m_v.size() * sizeof (short)) {
return *((char*) m_v.data() + m_offset++);
} else {
return -1;
}
}
int m_offset;
std::vector<short> m_v;
};
Writer should collect output data of the Reader. If you want to collect it in char array I could recommend to do it like this.
struct writer : public libzpaq::Writer {
void put(int c) {
m_buffer.push_back(c);
}
int size() {
m_buffer.size();
}
void copy_to(char* dst) {
memcpy(dst, m_buffer.data(), m_buffer.size());
}
std::vector<char> m_buffer;
};
Then call it:
writer w;
reader r(v1);
libzpaq::compress(&r, &w, "5");
char* buffer = new char[w.size()];
w.copy_to(buffer);
If you want to use StringBuffer then you should write some data to buffer, before read, that why it returns 0. Look at example:
char* buffer = reinterpret_cast<char*> (&v[0]);
int length = sizeof (short)*v.size();
libzpaq::StringBuffer in, out1, out2;
// fill buffer with source data
in.write(buffer, length);
// compress to out1
libzpaq::compress(&in, &out1, "5");
// decompress out1 to out2
libzpaq::decompress(&out1, &out2);
// check result
short* b = (short*)out2.data();
for(int i = 0; i < 15; ++i) {
std::cout << b[i] << std::endl;
}

C++ reading some bytes at a memory address, output as string

A function returns a pointer and a length (via the arguments) from an unknown DLL.
Result = SpamnEggs( &pBytes, &nBytes )
The pointer points to a valid memory address at which are nBytes sequential bytes.
These bytes contain valid ascci values for text. There is no null termination!
I am tasked with "ovelaying" a string type of some sort in as few simple operations in generic C++ code (without complex libraries or using byte) before output:
cout << sresult
Added:
without copying the bytes as this is a large buffer that must be traversed.
Prototype:
int SpamnEggs( void* pBytes, void* nBytes );
becomes
int SpamnEggs( char** pBytes, int* nBytes );
Many thanks all. Great answers and all very valid.
You can copy the raw memory and add the string terminating character yourself:
char* newStr = new char[size + 1];
memcpy ( newStr, source, size );
newStr[size] = "\0";
cout << newStr;
Without copying memory, you can create a class that holds the pointer and length as members and overload the stream operator to print only length characters:
class MyString
{
void* _pBuf;
int _length;
public:
MyString(void* pBuf, int length)
{
_pBuf = pBuf;
_length = length;
}
friend ostream& operator <<(ostream &os,const MyString& obj);
};
ostream& operator <<(ostream &os,const MyString& obj)
{
const char* pChar = (const char*)obj._pBuf;
for ( int i = 0 ; i < obj._length ; i++ )
{
os << pChar[i];
}
return os;
}
Example usage:
char* x = "blablabla";
int length = 3;
MyString str(x,length);
cout << str;
You can just construct a std::string from the pointer and a length.
std::string sResult(pBytes, nBytes);
std::cout << sResult;
(assuming pBytes is a char* pointer, otherwise you need a small cast).
How about something like this (untested) code:
class my_pointer_string
{
friend std::ostream &operator<<(std::ostream &, const my_pointer_string &);
public:
my_pointer_string(void *ptr, size_t len)
: m_pointer(ptr), m_length(len)
{ }
private:
void *m_pointer;
size_t m_length;
};
std::ostream &operator<<(std::ostream &os, const my_pointer_string &str)
{
char *string = reinterpret_cast<char *>(str.m_pointer);
for (size_t i = 0; i < str.m_length; i++)
os << *string++;
return os;
}
What you would have to do is
a) Create some class that encapsulates the char pointer and the size.
b) Write a << operator for that class to output its content to a stream.
EDIT: In contrast to the response by Bo Persson this would not imply copying the source data.