I am working on a video streaming application in C++, and I have questions about serialization of the packets. I know there are a number of serialization frameworks out there from Google Protocol Buffers, Apache Thrift, Cereal, etc. I am correct in thinking that the packets will need to be serialized, right?
Since any little bit of overhead can cause latency, is it worthwhile to use an existing framework to do the serialization?
Would it even be worth my time to try and roll my own or just stick to the frameworks?
I know most of the frameworks will take into account big-endian and little-endian which is nice.
From the research I have done, I was leaning towards trying to use Google Protocol Buffers. Is this a good serialization library for a streaming application?
Are there any other suggestions?
I appreciate the advice and suggestions.
The framework should help you send and retrieve packets. Filling in the packet with a payload is your task.
If you are worried about the latency and don't want to delay your processing by waiting for packets to be sent, you should consider using a separate thread and buffer for the packets.
For example, the processing thread would create or fill in a packet and put it into the outgoing buffer (e.g. circular queue). The packet sending thread would take packets from the buffer and send them out. The thread would sleep until either the packet has been sent or a new packet is added to the queue.
This technique will "blast" the network with packets, which you may not want to do. If you continuously blast packets, nobody else will be able to send out packets. This may not be a problem depend on routers and switches on the network and your packet framework.
Related
We decided to use UDP to send a lot of data like coordinates between:
client [C++] (using poll)
server [JAVA] [Apache MINA]
My datagrams are only 512 Bytes max to avoid as possible the fragmentation during the transfer.
Each datagram has a header I added (with an ID inside), so that I can monitor :
how many datagrams are received
which ones are received
The problem is that we are sending the datagrams too fast. We receive like the first ones and then have a big loss, and then get some, and big loss again. The sequence of ID datagram received is something like [1], [2], [250], [251].....
The problem is happening in local too (using localhost, 1 network card only)
I do not care about losing datagrams, but here it is not about simple loss due to network (which I can deal with)
So my questions here are:
On client, how can I get the best :
settings, or socket settings?
way to send as much as I can without being to much?
On Server, Apache MINA seems to say that it manage itself the ~"size of the buffer socket"~ but is there still some settings to care about?
Is it possible to reach something like 1MB/s knowing that our connection already allow us to have at least this bandwidth when downloading regular files?
Nowadays, when we want to transfer a ~4KB coordinates info, we have to add sleep time so that we are waiting 5 minutes or more to get it to finish, it's a big issue for us knowing that we should send every minute at least 10MB coordinates informations.
If you want reliable transport, you should use TCP. This will let you send almost as fast as the slower of the network and the client, with no losses.
If you want a highly optimized low-latency transport, which does not need to be reliable, you need UDP. This will let you send exactly as fast as the network can handle, but you can also send faster, or faster than the client can read, and then you'll lose packets.
If you want reliable highly optimized low-latency transport with fine-grained control, you're going to end up implementing a custom subset of TCP on top of UDP. It doesn't sound like you could or should do this.
... how can I get the best settings, or socket settings
Typically by experimentation.
If the reason you're losing packets is because the client is slow, you need to make the client faster. Larger receive buffers only buy a fixed amount of headroom (say to soak up bursts), but if you're systematically slower any sanely-sized buffer will fill up eventually.
Note however that this only cures excessive or avoidable drops. The various network stack layers (even without leaving a single box) are allowed to drop packets even if your client can keep up, so you still can't treat it as reliable without custom retransmit logic (and we're back to implementing TCP).
... way to send as much as I can without being to much?
You need some kind of ack/nack/back-pressure/throttling/congestion/whatever message from the receiver back to the source. This is exactly the kind of thing TCP gives you for free, and which is relatively tricky to implement well yourself.
Is it possible to reach something like 1MB/s ...
I just saw 8MB/s using scp over loopback, so I would say yes. That uses TCP and apparently chose AES128 to encrypt and decrypt the file on the fly - it should be trivial to get equivalent performance if you're just sending plaintext.
UDP is only a viable choice when any number of datagrams can be lost without sacrificing QoS. I am not familiar with Apache MINA, but the scenario described resembles the server which handles every datagram sequentially. In this case all datagrams arrived while the one is serviced will be lost - there is no queuing of UDP datagrams. Like I said, I do not know if MINA can be tuned for parallel datagram processing, but if it can't, it is simply wrong choice of tools.
I know how to open an UDP socket in C++, and I also know how to send packets through that. When I send a packet I correctly receive it on the other end, and everything works fine.
EDIT: I also built a fully working acknowledgement system: packets are numbered, checksummed and acknowledged, so at any time I know how many of the packets that I sent, say, during the last second were actually received from the other endpoint. Now, the data I am sending will be readable only when ALL the packets are received, so that I really don't care about packet ordering: I just need them all to arrive, so that they could arrive in random sequences and it still would be ok since having them sequentially ordered would still be useless.
Now, I have to transfer a big big chunk of data (say 1 GB) and I'd need it to be transferred as fast as possible. So I split the data in say 512 bytes chunks and send them through the UDP socket.
Now, since UDP is connectionless it obviously doesn't provide any speed or transfer efficiency diagnostics. So if I just try to send a ton of packets through my socket, my socket will just accept them, then they will be sent all at once, and my router will send the first couple and then start dropping them. So this is NOT the most efficient way to get this done.
What I did then was making a cycle:
Sleep for a while
Send a bunch of packets
Sleep again and so on
I tried to do some calibration and I achieved pretty good transfer rates, however I have a thread that is continuously sending packets in small bunches, but I have nothing but an experimental idea on what the interval should be and what the size of the bunch should be. In principle, I can imagine that sleeping for a really small amount of time, then sending just one packet at a time would be the best solution for the router, however it is completely unfeasible in terms of CPU performance (I probably would need to busy wait since the time between two consecutive packets would be really small).
So is there any other solution? Any widely accepted solution? I assume that my router has a buffer or something like that, so that it can accept SOME packets all at once, and then it needs some time to process them. How big is that buffer?
I am not an expert in this so any explanation would be great.
Please note, however, that for technical reasons there is no way at all I can use TCP.
As mentioned in some other comments, what you're describing is a flow control system. The wikipedia article has a good overview of various ways of doing this:
http://en.wikipedia.org/wiki/Flow_control_%28data%29
The solution that you have in place (sleeping for a hard-coded period between packet groups) will work in principle, but in order to get reasonable performance in a real-world system you need to be able to react to changes in the network. This means implementing some kind of feedback where you automatically adjust both the outgoing data rate and packet size in response to to network characteristics, such as throughput and packetloss.
One simple way of doing this is to use the number of re-transmitted packets as an input into your flow control system. The basic idea would be that when you have a lot of re-transmitted packets, you would reduce the packet size, reduce the data rate, or both. If you have very few re-transmitted packets, you would increase packet size & data rate until you see an increase in re-transmitted packets.
That's something of a gross oversimplification, but I think you get the idea.
I'm working on reliable UDP networking and I have to know something. I think UDP reliable algorithm works like that (IDK, I guess);
Server send: (header:6)abcdef
Client receive: (header:6)abdf, sends back "I got 4 data, they are abdf"
Server send: (header:2)ce
Client receive: (header:2)ce, OK I'm going to combine them!
Now is this true way to reliable UDP?
EDIT (after answer, maybe this can be helpful for someone): I'm goint to use TCP because reliable UDP is not a good way to handle my operations. I'll send position like un-important, temporal variables. Maybe if I create a algorithm for reliable UDP, this reliable process will took 3-4 UDP send-recv that means I can send 3-4 other unreliable position data at this time and I'm sending small datas which is can be more efficiency than reliable UDP.
The "true way" to get reliable UDP is to use TCP.
If you still want to do it over UDP, you can verify the integrity of the message by sending a checksum with the message, and then recalculating the checksum at the other end to see if it matches the checksum you sent.
If it doesn't match, request the packet again. Note that this is essentially reinventing TCP.
Well, even with:
- Client receive: (header:6)abdf, sends back "I got 4 data, they are abdf"
- Server send: (header:2)ce
what if server will not receive your response (which may happen in UDP)? So switching to TCP is much better option, if you're not concerned about speed of connection.
Your problem sounds like it's tailor-made for the Data Distribution Service.
I'll send position like un-important, temporal variables
In fact, location ordinates are the popular examples for many of its vendors. RTI has a demonstration which goes well with your use case.
Yeah, a lot of folks groan when they hear "IDL" but I'd recommend that you give it a fair shake. DDS is unlike very many popular pub-sub/distribution/etc protocols in that it's not a simple encapsulation/pipeline.
I think the really cool thing is that often a lot of logic and design elements go into the problem of "how do I react the underlying network or my peer(s) misbehave(s)?" DDS offers a quality of service negotiation and hooks for your code to react when the QoS terms aren't met.
I would recommend against taking this decision lightly, it's a good deal more complex than TCP, UDP, AMQP, etc. But if you can afford the complexity and can amortize it over a large enough system -- it can pay real dividends.
In the end, DDS does deliver "reliable" messages over UDP. It's designed to support many different transports, and many different dimensions of QoS. It's truly dizzying when you see all the different dimensions of QoS that are considered by this service.
I am writing a protocol to transfer gigabytes of data over a network using TCP, to try to teach myself a little bit about programming on protocols. I am unsure of how to design this transfer protocol, in order to transfer the data in the fastest and most efficient way.
I am using Qt on windows.
At the moment, my design of my application protocol (the part to transfer the data) is as follows:
First shoot the login details.
Write the first data packet (into the socket) of 4 kilobytes, and then wait for the server to confirm it has got the packet.
When the server confirms receiving the data packet (by writing int "1"), write the next 4 kilobytes.
When all data has been transferred, send the md5sum of the data transferred to the server.
If the server confirms again with an int 8, data transfer completes.
At the moment, I am not able to get speeds higher than 166KB/sec on the same computer when transferring over 127.0.0.1. I have been trying to read other protocol designs, but there is hardly any documentation on data transfer protocols that one can write for their application.
Is the protocol design that I've posted wrong or suffering from some serious issues?
Should the protocol wait for each packet to be confirmed by the server or should I write it continuously?
First, I would recommend spending some time on reading about TCP, and about Sliding Window Protocol.
I think there are 2 reasons why your implementation is so slow: first, you wait for acknowledgement of each packet - very slow, you should use sliding window.
Second, you use MD5 checksumming. There is nothing wrong with that, but TCP already implements some basic checksumming, and MD5 implementation you use can be very slow.
And finally, typical way to find out why something works very slow is to use profiling.
I have a TCP connection opened between Symbian and a Server machine and I would like
to transfer huge chunks of data (around 32K) between these two endpoints. Unfortuantely,
the performance figures are pretty poor and I am looking for ideas how I could improve
my implementation. One of the things I tried was to increase the number of bytes that can be
buffered by the socket for sending & receiving to 64K.
iSocket.SetOpt(KSoTcpSendWinSize, KSolInetTcp, 0x10000);
iSocket.SetOpt(KSoTcpRecvWinSize, KSolInetTcp, 0x10000);
Are there any other things that could be optimized at a socket level for better throughput?
It is also possible, that my socket code does something stupid. It follows a simple request/response
protocol. I have to use the blocking WaitForRequest routine to be sure that the data has been send/received
so that I can process it then.
//store requestinfo in reqbuf and send it to server; wait for iStatus
iSocket.Send( reqbuff, 0, iStatus, len );
User::WaitForRequest(iStatus);
//store 32K file in resbuff; wait for iStatus to be sure that all data has
//been received
iSocket.Recv(resbuff, 0, iStatus, len);
User::WaitForRequest(iStatus);
//do something with the 32K received
Would be thankful for every comment!
You can send and receive in parallell if you use active objects. There should be example code in the SDK. Obviously it depends on the application and protocol used whether that will help.
I'm no TCP expert, but I think there are parameters on the socket that can cause your usage pattern (sending one large buffer, then receiveing a large buffer) to not use the network optimally compared to when sending approximately equal amounts of data in both directions.
All things about TCP sockets that can be configured in other OS:se should be possible to configure on Symbian as well, but first you need to figure out what. I suggest you ask another question that is TCP general and get some pointers. Then you can figure out how to set that up in Symbian.
Are you positive that the
//do something with the 32K received
doesn't take particularily long? Your app appears to be single-threaded so if this is holding up the line, well thats an obvious bottleneck.
Also what do you mean by poor performance? Have you compared the performance to other tcp apps?
Lastly, if performance is a big issue you can switch over to raw sockets/datagram sockets, and optimize your own validation protocol for your specific data.