My program uses UDP to send data between two programs, it works great however I have added a new vector into the data I want to send, the vector type is another class which looks like this...
class Bullet: public Sprite
{
public:
float speed;
};
The DataPacket...
typedef struct DataPacket
{
int ID; //Player ID
int elapsedTime; //Total elapsed player time
float x, y; //X & Y pos of player
std::vector<Bullet>* pBullets; //Vector containing all the players bullets
};
Is there a way to send this data correctly? The expression cannot be evaluated once the server receives the data from the client, every other part is correct in the received packet.
Basically the server is receiving the positional data of the bullets on the screen which is contained in this Bullet class along with a few other items.
Just to note: pBullet never used to be a pointer but in an attempt to try and figure out what was wrong I changed it to a pointer...it never fixed the issue tho
You cannot do this. A vector is a class which is typically implemented with an internal pointer to dynamic memory which changes location as the vector grows.
When you try and serialize the vector by casting your whole structure, you just serialize the pointer to the memory holding the vector contents, you don't get the contents itself because it's not part of that structure.
You'll have to individually serialize all items in the vector one-by-one and individually add them back in when you deserialize it.
Note that you can change the vector of bullets to a statically sized array internal to the structure and then it would be contiguous in memory and you could just serialize the whole structure - and you can also make the last element of the array an array of one bullet, and then allocate memory for the structure size + (x-1)*sizeof(Bullet), allowing you to overwrite the array of 1 for all of the extra memory you added. This would also be contiguous in memory allowing you to serialize the whole memory region pretty easily.
You should also look in to htonl, ntohl, htons, ntohs and start network-byte-ordering your data as well if you're going to send it over a network to keep the byte endianness from being an issue on some other systems you might end up using.
Use Serialization whenever you have to send data across the network. You may refer to Boost Serialization.
Your DataPacket is something whose size is dynamic. Had it been an array than vector, it might have worked given the same endianness of the machines.
For Game Development I would recommend to use Google ProtoBuf. In comparison with Boost Serialization, it provides binary serialization (Boost Serialization has an example of binary serialization but it's not very portable) and is more convenient for complex data structures.
You need to serialize the data yourself. You can't just send std::vector or pointer thereto.
Related
I'm currently using google's protobuffer library to store and load data from disk.
It's pretty convenient because it's fast, provides a nice way of defining my datastructures, allows to compress/decompress the data upon writing/reading from the file.
So far that served me well. The problem is now that I have to deal with a datastructure that is several hundred gigabytes large and with protobuf I can only write and load the whole file.
The datastructure looks something like that
struct Layer {
std::vector<float> weights_;
std::vector<size_t> indices_;
};
struct Cell {
std::vector<Layer> layers_;
};
struct Data {
int some_header_fields;
...
std::vector<Cell> cells_;
};
There are two parts to the algorithm.
In the first part, data is added (not in sequence, the access pattern is random, weights and indices might be added to any layer of any cell). No data is removed.
In the second part, the algorithm accesses one cell at a time and processes the data in it, but the access order of cells is random.
What I'd like would be something similar to protobuf, but backed by some file storage that doesn't need to be serialized/deserialized in one go.
Something that would allow me to do things like
Data.cells_[i].layers_[j].FlushToDisk();
at which point the weights_ and indices_ arrays/lists would write their current data to the disk (and free the associated memory) but retain their indices, so that I can add more data to it as I go.
And later during the second part of the algorithm, I could do something like
Data.cells_[i].populate(); //now all data for cell i has been loaded into ram from the file
... process cell i...
Data.cells_[i].dispose(); //now all data for cell i is removed from memory but remains in the file
Additionally to store data to the disk, I'd like it to also support compression of data. It should also allow multithreaded access.
What library would enable me to do this? Or can I still use protobuf for this in some way? (I guess not, because I would not write the data to disk in a serialized fashion)
//edit:
performance is very important. So when I populate the cell, I need the data to be in main memory and in continguous arrays
Before anything, I want to say the title is not the question.
My question is why can't you just send all the bytes of the structure and then cast it into that structure (giving you have the structure defined in both sides, which makes sense you have).
Thank you!
EDIT: Here's my current structure:
struct COMPUTER_INFO
{
const char* Name;
int Brightness;
int Volume;
}
I was thinking that it you can easily calculate the size of all that and then send it trough send().
Name is a pointer (contains an address) that only makes sense to your program on your computer. If you sent this structure as bytes the receiving program would receive just the address not the characters that comprise the name. The received address would also not point to a valid location in memory in the receiving computer.
Brightness and Volume are ints - ints do not have a fixed size they are the "natural" word size of the computer (the standard does impose some restrictions). So the sizeof(int) on the sending and receiving computer may be different. There may also be encoding differences e.g. big vs little endian. See: https://en.wikipedia.org/wiki/Endianness
in general you are right. You can, of course, send the raw byte streams. But how do you want the receiver to recognize the positions of the members of your structure in this byte stream? Especially, the char* in your example is of variable size, so this will not be possible.
I recommend using the Boost serialization package. You can find a detailed tutorial here. The package will take care of "serializing" (packaging your object into a byte stream) and "deserializing" (constructing your object from the byte stream). It is absolutely convenient for nearly all STL containers and can easily be expanded for custom types.
I want to send a vector from one computer to another through internet. I'm looking for a Peer-to-Peer solution in C++. I have made a Winsock2 solution, but it can only send char* with the send and recv-functions which I don't think will work for my project.
Is there a way of using JSON with a P2P-solution in C++? So make a JSON-object of the vector and send it through internet? Or do you know a better solution?
The vector I want to send through internet to another computer looks like this:
Vector<AVpacket> data;
AVpacket is a struct from ffmpeg, consisting 14 data members. https://ffmpeg.org/doxygen/trunk/structAVPacket.html. You don't want to make this to a char*
You can actually send anything using the send and recv functions. You just have to make sure you pass a pointer to the data, and then typecast that pointer as a char * and it will work.
However, you can't send a std::vector as is. Instead you should first send its size (otherwise the receiving end will not know how much data it should receive) then you send the actual data in the vector, i.e. someVector.data() or &someVector[0].
Though in your case it will be even more complicated, as the structures you want to send contains embedded pointers. You can't send pointers over the Internet, it's barely possible to transfer pointers between two processes on the same system. You need to read about serialization and maybe about the related subject marshalling.
In short: You can send any kind of data, it doesn't have to be characters, and for the kind of structures you want to send you have to convert them to a format is transferable through serialization.
You can not simply send a vector. You can not be sure how the std allocator reserved the memory. So it is very likely that the whole memory of the vector is not just one big linear chunk.
In addition to this, as pointed out above, there are pointers in this struct. They point to data in your local memory. These addresses aren't valid on the recipients side, thus you would access invalid memory trying to read this.
I guess, that in order to achieve what you want to do, you have to try a completely different approach. Do not get lost by trying to send the data of the pointers or similar, rather try having parallel data on both machine.
E.g. both load the same video e.g. from a file which both share. Then use a unique identifier for that video to reference the video on both sides.
I'm working to copy the following structure to a byte array to send over a named pipe. I've found that since switching from a byte array that I had given a static definition, to a vector because my host length will be of varying lengths.
Here is the outline of my structure:
USHORT version; // Header Version
USHORT type; // IPVersion
USHORT count; // Number of IP addresses of remote system
USHORT length; // Header Length (1)
BYTE SysConfigLocIP[4];
BYTE SysConfigRemoteIP[4];
USHORT lengthHost;
std::vector<BYTE>HostName;
later, after filling the structure I copy it to a byte like so:
BYTE Response[sizeof(aMsg)]
memcpy(response, &aMsg, sizeof(aMsg))
I find that my array is vector is holding the correct information for the host when I inspect the container during a debug. However, after the copy to the Response byte array, I'm finding the data that has been copied is drastically different. Is this a valid operation, if so, what can I do correctly copy the data from my vector the BYTE array. If not, what are other strategies I can use to dynamically size the structure to send the hostnames? Thank you for taking the moment of time to read my question, and I appreciate any feedback.
I'm working to copy the following structure to a byte array to send
over a named pipe.
named pipe (or other forms of inter-process or inter-processor communication) does not understand your struct, neither do they understand vector. They just operate on the concept of byte-in-byte-out. It is up to you, the programmer, to assign meaning to those bytes.
As suggested, please read on serialization. Try starting at http://en.wikipedia.org/wiki/Serialization. If permitted you can use the Boost solution, http://www.boost.org/doc/libs/1_55_0/libs/serialization/doc/index.html, but I would still encourage you to understand the basics first.
As an exercise, first try transferring a vector<int> from sender to receiver. The number of elements in the vector must not be implicitly known by the receiver. Once you achieve that, migrating from int to your struct would be trivial.
That memcpy will only work for POD (plain old data) types. A vector is not POD. Instead, write code to put each byte in the buffer exactly where it needs to be. Don't rely on "magic".
99% of the time in C++ there is no reason to use memcpy. It breaks classes. Learn about copy constructors and std::copy and use them instead.
I'm using C++ and wondering if I can just send an entire int array over a network (using basic sockets) without doing anything. Or do I have to split the data up and send it one at a time?
Yes.
An array will be laid out sequentially in memory so you are free to do this. Simply pass in the address of the first element and the amount of data and you'll send all data.
You could definitely send an array in one send, however you might want to do some additional work. There are issues with interpreting it correctly at the receiving end. For example, if using different machine architectures, you may want to convert the integers to network order (e.g., htonl).
Another thing to keep in mind is the memory layout. If it is a simple array of integers, then it would be contiguous in memory and a single send could successfully capture all the data. If, though, (and this is probably obvious), you have an array with other data, then the layout definitely needs consideration. A simple example would be if the array had pointers to other data such as a character string, then a send of the array would be sending pointers (and not data) and would be meaningless to the receiver.