Serialization of struct - c++

Suppose i have a struct whose member values i want to send over the network to another system using winsock 2. I'm using C++ language.
How do i convert it to char * keeping in mind that the struct has to be serialized before sending and also how do i deserialize the char * into struct at the other end? I found boost serialization as a suggestion to similar question but can anyone illustrate with a small code snippet for both serialization and deserialization ?
This question might seem very basic but the other answers to the related posts did not help much.

Following example shows a simplest way to serialize struct into char array and de-serialize it.
#include <iostream>
#include <cstring>
#define BUFSIZE 512
#define PACKETSIZE sizeof(MSG)
using namespace std;
typedef struct MSG
{
int type;
int priority;
int sender;
char message[BUFSIZE];
}MSG;
void serialize(MSG* msgPacket, char *data);
void deserialize(char *data, MSG* msgPacket);
void printMsg(MSG* msgPacket);
int main()
{
MSG* newMsg = new MSG;
newMsg->type = 1;
newMsg->priority = 9;
newMsg->sender = 2;
strcpy(newMsg->message, "hello from server\0");
printMsg(newMsg);
char data[PACKETSIZE];
serialize(newMsg, data);
MSG* temp = new MSG;
deserialize(data, temp);
printMsg(temp);
return 0;
}
void serialize(MSG* msgPacket, char *data)
{
int *q = (int*)data;
*q = msgPacket->type; q++;
*q = msgPacket->priority; q++;
*q = msgPacket->sender; q++;
char *p = (char*)q;
int i = 0;
while (i < BUFSIZE)
{
*p = msgPacket->message[i];
p++;
i++;
}
}
void deserialize(char *data, MSG* msgPacket)
{
int *q = (int*)data;
msgPacket->type = *q; q++;
msgPacket->priority = *q; q++;
msgPacket->sender = *q; q++;
char *p = (char*)q;
int i = 0;
while (i < BUFSIZE)
{
msgPacket->message[i] = *p;
p++;
i++;
}
}
void printMsg(MSG* msgPacket)
{
cout << msgPacket->type << endl;
cout << msgPacket->priority << endl;
cout << msgPacket->sender << endl;
cout << msgPacket->message << endl;
}

You can also have a look at Protocol Buffers from Google which is a platform/language independent library for sending data between hosts.
However, the paradigm is shifted towards writing the protocol first and then fitting your data structures into it. The advantage of this though is that it forces your software architecture to fit well with simple data types.

You can just do
struct MyStruct {
int data;
char* someNullTerminatedName; // Assuming not larger than 1023 chars
std::ostream& serialize(std::ostream& os) const {
char null = '\0';
os.write((char*)&data, sizeof(data));
os.write(someNullTerminatedName, strlen(someNullTerminatedName));
os.write(&null, 1);
return os;
}
std::istream& deserialize(std::istream& is) {
char buffer[1024];
int i = 0;
is.read((char*)&data, sizeof(data));
do { buffer[i] = is.get(); ++i; } while(buffer[i] != '\0');
if (someNullTerminatedName != NULL) free(someNullTerminatedName);
someNullTerminatedName = (char*)malloc(i);
for (i = 0; buffer[i] != '\0'; ++i) {
someNullTerminatedName[i] = buffer[i];
}
return is;
}
};
It's up to you to take care of endianness and differences in size of ints and whatnot.
Example:
MyStruct foo, bar;
std::stringstream stream;
foo.serialize(stream);
// ... Now stream.str().c_str() contains a char* buffer representation of foo.
// For example it might contain [ 1f 3a 4d 10 h e l l o w o r l d \0 ]
bar.deserialize(stream);
// ... Now bar is a copy, via a serial stream of data, of foo.
If you have a socket library that exposes its interface via C++ iostreams then you don't even need the stringstream.

If your struct is POD you can use memcpy:
::memcpy(data, &your_struct, sizeof(YourStruct)));
and vice versa at reception:
::memcpy(&your_struct, data, sizeof(YourStruct)));
Where data is a char*. Don't forget you have to Allocate it, making sure it's big enought and delete it at the end.

Ok I will take the example from the boost web site as I don't understand what you can not understand from it.
I added some comments and changes to it how you can transfer via network. The network code itself is not in here. For this you can take a look at boost::asio.
int main() {
// create and open a character archive for output
// we simply use std::strinstream here
std::stringstream ofs;
// create class instance
const gps_position g(35, 59, 24.567f);
// save data to archive
{
boost::archive::text_oarchive oa(ofs);
// write class instance to archive
oa << g;
// archive and stream closed when destructors are called
}
// now we have const char* ofs.str().c_str()
// transfer those bytes via network
// read them on the other machine
gps_position newg;
{
// create and open an archive for input
std::stringstream ifs(the_string_we_read_from_the_network);
boost::archive::text_iarchive ia(ifs);
// read class state from archive
ia >> newg;
// archive and stream closed when destructors are called
}
return 0;
}

Related

Saving data to an array of pointers when reading form an file

I do not understand why my array of pointers is only saving the last line from the file that I am reading from. When I substitute a string literal into the setData() function the code works just fine. All that the "mann" file contains are a bunch of words order alphabetically. Thank you.
#include <iostream>
#include <fstream>
using namespace std;
class orignialData {
char* data;
public:
void setData(char* s) { data = s;}
char* getData() const {return data;}
};
class dataClass {
orignialData** W_;
public:
dataClass(char* filename);
void addData();
void viewAll();
};
dataClass::dataClass(char* filename) {
fstream file;
file.open(filename, ios::in);
if (file.fail()) {
cout << "There was an error reading the file...\n";
}
W_ = 0;
W_ = new orignialData*[5];
for (int i = 0; i < 5; i++)
W_[i] = new orignialData;
char buff[30];
char* temp;
while(file >> buff) {
cout << buff << endl;
static int i = 0;
W_[i] -> setData(buff);
i++;
}
file.close();
}
Instead of data = s, write data = strdup(s) to make a copy of the contents. Otherwise, you will assign the same pointer again and again, and you will overwrite the contents of the memory to which this pointer points again and again. At the end, your temporary buffer will contain the last line of your file, and all the pointers will point to exactly this buffer. That's what you are observing...

C++ read binary file char by char

I have an assignment of saving some objects data in a specific order of its data members, I'll try to simplfy . Consider this Base class constructor from a binary file. (Please note that it was not my choice to use char *) .
Base(ifstream & in_file) {
int n;
in_file.read((char *)&n, sizeof(n));
m_var = new char[n + 1];
in_file.read(m_var, n);
m_var[n] = '\0';
in_file.read((char *)&m_intvar, sizeof(m_intvar));
}
It has to initialize m_var (char *) and another integer variable. This code works, though it requiers to save the length of the char * for me to allocate the memory.
The problem starts here. I was instructed not to save the size of the string, but to only enter a \n after each value i write to the file. So I need some how to read the file, and get the string until the \n character.
I was thinking about reading char by char, but couldn't find a way to do it, I assume there is an istream function that offers that. Some similar function to >> of a text file would also be good I assume.
After consulting cppreference.com I end up as follows:
#include <fstream>
#include <iostream>
#include <cstring>
class Base
{
public:
Base(std::istream & in_file) { // NOTE: changed to istream to allow reading from any stream not just files
in_file.read((char *)&n, sizeof(n));
char buffer[1024];
in_file.get(buffer, sizeof(buffer), '\n');
size_t gcount = in_file.gcount();
if (in_file.get() != '\n')
{
throw std::runtime_error("binary string to long"); // you may want to implement a loop here using peek() to check for newline
}
m_var = new char[gcount];
std::copy(buffer, buffer + gcount, m_var);
}
Base(int num, const char* strg)
: n(num)
, m_var(strdup(strg))
{
}
bool operator == (const Base& rhs)
{
return n == rhs.n && strcpy(m_var, rhs.m_var);
}
void write(std::ostream& out)
{
out.write((char*)&n, sizeof(n));
out.write(m_var, strlen(m_var));
out.write("\n", 1);
}
int n;
char* m_var = nullptr;
~Base()
{
delete m_var;
}
};
int main(int, char**)
{
Base b1(10, "Hello Word!");
{
std::ofstream out("testfile.bin");
b1.write(out);
}
std::ifstream in("testfile.bin");
Base b2(in);
if (b1 == b2)
{
std::cout << "read ok!" << std::endl;
}
else
{
std::cout << "read failed!" << std::endl;
}
return 0;
}
4 years later but I was having a similar problem and found this post.
You can read char by char as you mention, with a loop.
int i;
for(i=0;;i++){
cout<<i;
in_file.read(&buffer[i],sizeof(char));
if buffer[i]=='\n') break;
}
The only problem with the code I came up is it saves the '\n'. But you can and should replace it to the NULL char '\0' after having found the new line '\n'.
(buffer[i]=='\0')
Please correct me if I am mistaken.

How to serialize a vector into an array of chars

I have a vector defined as:
std::vector<message *>
where message is:
struct message{
static unsigned int last_id;
unsigned int id;
std::string msg;
std::string timestamp;
}
My objective is to send this information using Winsock (from server to client), but this only allows sending chars as it appears in WinSock2.h. Taking this into account, I want to serialize all the information (id, msg and timestamp) in an array of chars in order to send it all together, and in the client, have a function to deserialize so as to have the same vector I had in the server.
How could I implement it?
Any help is appreciated.
The following offers a simple approach for the serialization problem.
However, note that it is not portable. It assumes same environment conditions on both sides (client/server), i.e. endianness and sizeof int and size_t. This assumption is probably unsatisfactory when writing server/client programs, and your code should handle this aspect as well.
For example, if you can say that 32 bits is a sufficient size for the id value and the length of your strings, you can use htonl when serializing, and ntohl when deserializing.
Serializer:
class MessageSerializer
{
public:
MessageSerializer(const message& messageStruct)
: m_msgRef(messageStruct)
, m_msgLength(m_msgRef.msg.length())
, m_timeLength(m_msgRef.timestamp.length())
{}
size_t RequiredBufferSize() const
{
return sizeof(int) + sizeof(size_t)*2 + m_msgLength + m_timeLength;
}
void Serialize(void* buffer) const
{
PushNum (buffer, m_msgRef.id);
PushString (buffer, m_msgRef.msg.c_str(), m_msgLength);
PushString (buffer, m_msgRef.timestamp.c_str(), m_timeLength);
}
private:
const message& m_msgRef;
const size_t m_msgLength;
const size_t m_timeLength;
template<typename INTEGER>
void PushNum(void*& buffer, INTEGER num) const
{
INTEGER* ptr = static_cast<INTEGER*>(buffer);
//copying content
*ptr = num;
//updating the buffer pointer to point the next position to copy
buffer = ++ptr;
}
void PushString(void*& buffer, const char* cstr, size_t length) const
{
PushNum(buffer, length);
//copying string content
memcpy(buffer, cstr, length);
//updating the buffer pointer to point the next position to copy
char* ptr = static_cast<char*>(buffer);
ptr += length;
buffer = ptr;
}
};
Deserializer:
class MessageDeserializer
{
public:
MessageDeserializer(const char* messageBuffer)
: m_msgBuffer(messageBuffer)
{}
void Deserialize(message& messageOut)
{
messageOut.id = PopNum<int>(m_msgBuffer);
messageOut.msg = PopString(m_msgBuffer);
messageOut.timestamp = PopString(m_msgBuffer);
}
private:
const void* m_msgBuffer;
template<typename INTEGER>
INTEGER PopNum(const void*& buffer) const
{
const INTEGER* ptr = static_cast<const INTEGER*>(buffer);
//copying content
INTEGER retVal = *ptr;
//updating the buffer pointer to point the next position to copy
buffer = ++ptr;
return retVal;
}
std::string PopString(const void*& buffer) const
{
size_t length = PopNum<size_t>(buffer);
const char* ptr = static_cast<const char*>(buffer);
//copying content
std::string retVal(ptr, length);
//updating the buffer pointer to point the next position to copy
ptr += length;
buffer = ptr;
return retVal;
}
};
Then your using code could be something like:
//...
MessageSerializer serializer(*myVector[i]);
char* buffer = new char[serializer.RequiredBufferSize()];
serializer.Serialize(buffer);
and:
//...
message myMsg;
MessageDeserializer(input).Deserialize(myMsg);
You can use the Boost serialization library to save/load your structure to an array of char. The boost library is widely used in C++, and if you're not familiar with it, I'd recommend taking a look at it.
Instead of using winsock, you could learn to use Boost sockets and make your C++ code work on almost any platform, instead of just Windows, but that's another topic.
Here's an example of how to serialize your vector, and recover it from the other side of the socket:
#include <vector>
#include <boost/serialization/nvp.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
struct message {
static unsigned int last_id;
unsigned int id;
std::string msg;
std::string timestamp;
template <class ArchiveT>
void serialize(ArchiveT& ar, const unsigned int /*version*/) // function used to serialize (save/load) data from the boost serialization library
{
ar & boost::serialization::make_nvp("LastId", last_id);
ar & boost::serialization::make_nvp("Id", id);
ar & boost::serialization::make_nvp("Msg", msg);
ar & boost::serialization::make_nvp("Timestamp", timestamp);
}
};
unsigned int message::last_id;
template <class T>
void serialize_save(const T& obj, std::string& outString)
{
std::stringstream binaryOut;
boost::archive::binary_oarchive outArchive(binaryOut);
outArchive << obj;
outString = binaryOut.str();
}
template <class T>
void serialize_load(T& dataOut, const void* data, const size_t dataSize)
{
const char* dataPtr = reinterpret_cast<const char*>(data);
std::string dataString(dataPtr, dataPtr + dataSize);
std::stringstream dataStream(dataString);
boost::archive::binary_iarchive binArchive(dataStream);
binArchive >> dataOut;
}
void init_vector(std::vector<message*>& vect) {
const size_t vectorSize = 2;
vect.resize(vectorSize);
for (size_t i = 0; i < vectorSize; i++) {
vect[i] = new message();
vect[i]->last_id = 0;
vect[i]->id = 1;
vect[i]->msg = "This is a message";
vect[i]->timestamp = "12:02pm";
}
}
int main() {
std::vector<message*> messages;
init_vector(messages); // initialize the vector. set it to any data
std::string outputBuffer;
serialize_save(messages, outputBuffer); // save the vector to a string (array of char)
socket_write(outputBuffer.c_str(), outputBuffer.size()); // write the serialized data to the socket
// on the reception side
std::string receiveBuffer;
socket_read(receiveBuffer); // receive socket data
std::vector<message*> receivedMessages;
serialize_load(receivedMessages, receiveBuffer.c_str(), receiveBuffer.size()); // from the array of character recover the vector
// here the vector receivedMessages contains the same values saved in init_vector()
}
You can change the export format if you'd like by changing the boost::archive::binary_iarchive object. For instance, replace it to boost::archive::xml_iarchive for serializing objects to XML. There are other formats provided by the library. Another advantage is that it supports versioning.

Reading a binary file in c++ with stringstream

I want to write/read data from a file. Is it possible to divide the file (inside the code) in multiple Strings/Sections? Or read data untill a specific line?
Just like: "Read the Data untill line 32, put it inside a String, read the next 32 lines and put it into another string"
Im already know how to read and find data with seekp but i dont really like it because my code always gets to long.
I already found some code but i dont understand it how it works:
dataset_t* DDS::readFile(std::string filename)
{
dataset_t* dataset = NULL;
std::stringstream ss;
std::ifstream fs;
uint8_t tmp_c;
try
{
fs.open(filename.c_str(), std::ifstream::in);
if (!fs)
{
std::cout << "File not found: " << filename << std::endl;
return NULL;
}
while(fs.good())
{
fs.read((char*)&tmp_c, 1);
if (fs.good()) ss.write((char*)&tmp_c, 1);
}
fs.close();
dataset = new dataset_t();
const uint32_t bufferSize = 32;
char* buffer = new char[bufferSize];
uint32_t count = 1;
while(ss.good())
{
ss.getline(buffer, bufferSize);
dataitem_t dataitem;
dataitem.identifier = buffer;
dataitem.count = count;
dataset->push_back(dataitem);
count++;
}
return dataset;
}
catch(std::exception e)
{
cdelete(dataset);
return NULL;
}
}
The Code edits a binary save file.
Or can someone link me a website where i can learn more about buffers and stringstreams?
You could create some classes to model your requirement: a take<N> for 'grab 32 lines', and a lines_from to iterate over lines.
Your lines_from class would take any std::istream: something encoded, something zipped, ... as long as it gives you a series of characters. The take<N> would convert that into array<string, N> chunks.
Here's a snippet that illustrates it:
int main(){
auto lines = lines_from{std::cin};
while(lines.good()){
auto chunk = take<3>(lines);
std::cout << chunk[0][0] << chunk[1][0] << chunk[2][0] << std::endl;
}
}
And here are the supporting classes and functions:
#include <iostream>
#include <array>
class lines_from {
public:
std::istream &in;
using value_type = std::string;
std::string operator*() {
std::string line;
std::getline(in, line);
return line;
}
bool good() const {
return in.good();
}
};
template<int N, class T>
auto take(T &range){
std::array<typename T::value_type, N> value;
for (auto &e: value) { e = *range; }
return value;
}
(demo on cpp.sh)

Store read memory process in a buffer then search it

I want to read the memory from a process for 16 MB (FFFFFF) and store it in a array, in a way that when I search inside the array like: array[i], i will be the real memory address.
Lets say I want to search from 000000 to FFFFFF, I want to make that jump sizeof(value), get the address from that address and store it in a var.
then if(var==value) return address.
i have this:
ReadProcessMemory(phandle,(void*)address,buffer,0xFFFFFF,0);
EDIT:
i have this (by BlueWanderer answer):
class offset_buffer{
private:
char *buf;
int offset;
public:
offset_buffer(char *in_buf, int in_offset)
: buf(in_buf), offset(in_offset){
}
char & operator[](int in_index){
return buf[in_index - offset];
}
void setOffset(int off){
offset=off;
}
void ReadMemory(){
LPBYTE point;
DWORD primeiroAddress = 0x000000;
DWORD finalAddress = 0xFFFFFF;
//LPBYTE buffer = new BYTE[finalAddress-primeiroAddress];
HANDLE phandle = OpenProcess(PROCESS_VM_READ,0,TargetPID);
ReadProcessMemory(phandle,(void*)primeiroAddress, buf, sizeof(buf), 0);
CloseHandle(phandle);
}
};
main(){
char *buffer = new char[0xFFFFFFF-0x0000000];
int address = 0x0000000;
offset_buffer b(buffer,address);
std::ostringstream ss;
int i=0;
TListItem *ListIt;
b.ReadMemory();
for(address=0x0000000;address<0xFFFFFFF;address+=sizeof(int)){
if(b[address]==StrToInt(Edit1->Text.c_str())){
ss << std::hex << address;
showValue();
ss.str(std::string());
}
}
what is wrong?? can someone help me? why it doesn't work
You want something like this?
class offset_buffer
{
private:
char *buf;
int offset;
public:
offset_buffer(char *in_buf, int in_offset)
: buf(in_buf), offset(in_offset)
{
}
char & operator[](int in_index)
{
return buf[in_index - offset];
}
};
It will map your real address to the index in the array
offset_buffer b(buffer, address);
if (b[0x0C2F8E3] == 123) return 0x0C2F8E3;