An interesting question I was asked in an interview:
Suppose you are constantly receiving some byte-stream from a source (let's assume a client-server model) with variable rate. You want to sort the packets on-the-fly at your end and retransmit them elsewhere at a constant rate. how would you implement such a system in C++?
I offered a basic system with a worker thread pushing packets into a heap and a dispatcher thread popping sorted packets and sending them away in sync with some internal clock in constant intervals of X.
The interviewer reasonably argued that such a system is prone to miss it's retransmission deadlines due to context switching between the threads. I replied that without any control of the thread scheduling algorithms in the specific machine, I can't guarantee constant retransmission rate. He followed with insistence that made me think that maybe I'm wrong and this is in fact achievable. So, am I?
Description of the problem is too vague. What is meant by "sorting"? Do packets have some sort of sequence numbers in their bodies? What if we never receive packet with some sequence number? Here are my thoughts on some generic algorithm that may be adapted for different specific situations.
Parameters
Algorithm depends on following parameters:
Max number of packets we are allowed to buffer (e.g. 4096).
High watermark value. It is a percentage relative to (1). If we currently have more packets in buffer than HWV, then we send packet even if it is in not sorted order. (50%, or 2048).
TTL -- max time we allow our packet to be buffered.
Variables
First of all, we need a ring buffer of fixed length.
Ring buffer consists of:
Array[4096] of buffered packets
Array[4096] of pointers into above array (to avoid copying packets themselves during sort operation)
Array[4096] of meta-information for each packet: its sequence number (parsed from packet) and timestamp of when packet was received
Pointers to head and tail.
We also need to store global variable -- next sequence number to send.
Algorithm
When packet arrives, we add our packet into buffer to the proper position (one step of insertion sort), so our buffer is always sorted by sequence number. After that, we may want to send packet from head of buffer if at least one of the following is true:
Seq number of head packet is less or equal than expected seq number. This means that head packet is next in sorted order. Typical situation.
Number of packets in buffer exceeds high watermark value (2048). We send head packet (despite the fact that it is not in sorted order) because we are afraid that burst of incoming activity may fill the rest of our buffer and we will have to throw away further incoming packets.
Current time minus time of arrival of head packet exceeds TTL.
If at least one of above is true, we send head packet and remove it from buffer. We also assign expected sequence number to be seq number of sent packet plus one. If, after sending head packet, buffer is not empty, we also (re)start timer with TTL value. When timer is fired, we perform the very same above checks. This is to avoid keeping packets indefinitely in buffer (in case there are no more incoming packets).
Related
I have to read some data (which is coming at a blinding speed - upto 5000 messages per second) from a multicast (UDP) stream. Because the stream is multicast (and the data is quite critical) the data provider has provided two streams that send identical data (their logic being that the possibility of the same packet dropping in both streams is very close to zero). All data packets are tagged with a sequence number to keep track.
Also, the application is so time critical that I am forced to listen to both streams in parallel and pick up the next sequence number from whichever multicast stream it was received on first - When the same packet comes on the mirror stream, I simply drop it.
I am planning to implement this drop feature using a common "sequence_number" variable between the two functions - which by the way run in different threads. The sequence number is atomic as it is going to be read and updated from two different threads.
The obvious algorithm that comes to mind is
if (sequence number received from the stream > sequence_number)
{
process packet;
sequence_number = sequence number received from the stream;
}
(The above algorithm needs to be modified for times when sequence numbers come out of order - and they can as it is a UDP stream - but lets forget about it for the time being)
My question is this:
From the time I std::load my sequence_number, check if it is smaller than the sequence number I have received from the stream, accept the packet, and finally std::store the new sequence number to sequence_number; if the other stream receives the same packet (with the same sequence number) and performs the same operations (before the first stream finishes std::store on that sequence number), I will essentially end up with the same packet twice in my system. What is a way to overcome this situation ?
Don't put off worrying about handling out of order packets until later, because solving that also provides the most elegant solution to synchronizing threads.
Elements of an array are unique memory locations for the purposes of data races. If you put each packet (atomically via pointer write) into a different array element according to its sequence number, you'll get rid of most of the contention. Also use compare-exchange to detect whether the other thread (other stream) has already seen that packet.
Note that you won't have the retry loop normally associated with compare-exchange, either you have the first copy of the packet and compare-exchange succeeds, or the packet already exists and your copy can be discarded. So this approach is not only lock-free but also wait-free :)
Here is one option, if you are using std::atomic values, using compare_exchange.
Not shown is how to initialize last_processed_seqnum, as you'll need to set it to a valid value, namely, one less than the seqnum of the next packet to arrive.
It will need to be adapted for the case in which there are sequence number gaps. You mention as part of your premise that there will be no dropped seqnums; but the example below will stop processing packets (i.e. fail catastrophically) upon any seqnum gaps.
std::atomic<int> last_processed_seqnum;
// sync last_processed_seqnum to first message(s).
int seqnum_from_stream = ...;
int putative_last_processed_seqnum = seqnum_from_stream - 1;
if (last_processed_seqnum.compare_exchange_strong(putative_last_processed_seqnum,
seqnum_from_stream))
{
// sequence number has been updated in compare_exchange_strong
// process packet;
}
Ideally, what we want is a compare_exchange function that uses greater than, not equals. I don't know of any way to achieve that behavior in one operation. The SO question I linked to links to an answer about iterating over all values less than a target to update.
You are probably implementing a price feed handler, which exchange is it and what protocol? Is it ITCH or FIX Fast? I would not recommend two threads for the same feed since you probably have to join several multicast groups for different market segments/boards.
I am trying to find a way to identify the start of a chunk of data sent via a TCP socket. The data chunk has the value of the integer 1192 written into it as the first four bytes, followed by the content length. How can I search the binary data (the char* received) for this value? I realize I can loop through and advance the pointer by one each time, copy out the first four bytes, and check it, but that isn't the most elegant or possibly efficient solution.
Is there also another way this could be done that I'm not thinking of?
Thanks in advance.
It sounds like linear scanning might be required, but you shouldn't really be losing your message positioning if the sending side of the connection is making its send()/write() calls in a sensible manner, you are reading in your buffers properly, and there isn't an indeterminate amount of "dead" space in the stream between messages.
If the protocol itself is sensible (there is at least a length field!), you should never lose track of message boundaries. Just read the marker/length pair, then read length payload bytes, and the next message should start immediately after this, so a linear scan shouldn't have to go anywhere ideally.
Also, don't bother copying explicitly, just cast:
// call htonl() to flip endianness if need be...
uint32_t x = *reinterpret_cast<uint32_t *>(charptr);
I am using the boost asio library to read some data using tcp. After using a.accept(*sock);, how to get the size of the 1st packet the client will send?
I use (sock->remote_endpoint().address()).to_string() to get the IP address of the user, so I guess there must be a similar simple way to get the size of the packet, right?
At the application level, it is often far more useful to know the number of bytes currently available for reading, rather than the packet size. The amount of data available for reading may be constructed from one or more TCP segments. In the OSI model, a TCP segment (Layer 4: Transport) may be constructed from one or more IP Layer packets (Layer 3: Network), and each packet may be constructed from one or more Ethernet frames (Layer 2: Data Link).
Therefore, I am going to assume the application is interested in knowing how many bytes to read, rather than knowing lower level details, such as the size of a packet. There are a few solutions to this problem:
Query the socket for how much data is available via socket::available(), then allocate the buffer accordingly.
std::vector<char> data(socket_.available());
boost::asio::read(socket_, boost::asio::buffer(data));
Use a class that Boost.Asio can grow in memory, such as boost::asio::streambuf. Some operations, such as boost::asio::read() accept streambuf objects as their buffer and will allocate memory as is required for the operation. However, a completion condition should be provided; otherwise, the operation will continue until the buffer is full.
boost::asio::streambuf data;
boost::asio::read(socket_, data,
boost::asio::transfer_at_least(socket_.available()));
As Igor R. suggests in the comments, incorporate length as part of the communication protocol. Check the Boost.Asio examples for examples of communication protocols. Focus on the protocol, not necessarily on the Boost.Asio API.
In a fixed length protocol, a constant byte size is used to indicate message boundaries, such as in the Boost.Asio Porthopper example. As the reader knows the size of the message, the reader can allocate a buffer in advance.
In a variable length protocol, such as the one used in the Boost.Asio Chat example, a message is often divided into two parts: a header and a body. One approach is to have a a fixed size header that contains various meta-information, such as the length of the body. This allows an application to read a header into a fixed size buffer, extract the body length, allocate a buffer for the body, then read the body.
// Read fixed header.
std::vector<char> data(fixed_header_size);
boost::asio::read(socket_, boost::asio::buffer(data));
protocol::header header(data);
network_to_local(header); // Handle endianess.
// Read body.
data.resize(header.body_length());
boost::asio::read(socket_, boost::asio::buffer(data));
protocol::body body(data);
network_to_local(body); // Handle endianess.
On the other hand, if I am mistaken, and you do need the total length of a packet, then one can use the basic_raw_socket. Boost.Asio's ICMP example demonstrates reading IPv4 packets from a socket, and extracting the header's field values.
I am reading data ~100 bytes at 100hz from a serial port. My buffer is 1024 bytes, so often my buffer doesn't get completely used. Sometimes however, I get hiccups from the serial port and the buffer gets filled up.
My data is organized as a [header]data[checksum]. When my buffer gets filled up, sometimes a message/data is split across two reads from the serial port.
This is a simple problem, and I'm sure there are a lot of different approaches. I am ahead of schedule so I would like to research different approaches. Could you guys name some paradigms that cover buffering in high speed data that might need to be put together from two reads? Note, the main difference I see in this problem from say other buffering I've done (image acquisition, tcp/ip), is that there we are guaranteed full packets/messages. Here a "packet" may be split between reads, which we will only know once we start parsing the data.
Oh yes, note that the data buffered in from the read has to be parsed, so to make things simple, the data should be contiguous when it reaches the parsing. (Plus I don't think that's the parser's responsibility)
Some Ideas I Had:
Carry over unused bytes to my original buffer, then fill it with the read after the left over bytes from the previous read. (For example, we read 1024 bytes, 24 bytes are left at the end, they're a partial message, memcpy to the beginning of the read_buffer_, pass the beginning + 24 to read and read in 1024 - 24)
Create my own class that just gets blocks of data. It has two pointers, read/write and a large chunk of memory (1024 * 4). When you pass in the data, the class updates the write pointer correctly, wraps around to the beginning of its buffer when it reaches the end. I guess like a ring buffer?
I was thinking maybe using a std::vector<unsigned char>. Dynamic memory allocation, guaranteed to be contiguous.
Thanks for the info guys!
Define some 'APU' application-protocol-unit class that will represent your '[header]data[checksum]'. Give it some 'add' function that takes a char parameter and returns a 'valid' bool. In your serial read thread, create an APU and read some data into your 1024-byte buffer. Iterate the data in the buffer, pushing it into the APU add() until either the APU add() function returns true or the iteration is complete. If the add() returns true, you have a complete APU - queue it off for handling, create another one and start add()-ing the remaining buffer bytes to it. If the iteration is complete, loop back round to read more serial data.
The add() method would use a state-machine, or other mechanism, to build up and check the incoming bytes, returning 'true' only in the case of a full sanity-checked set of data with the correct checksum. If some part of the checking fails, the APU is 'reset' and waits to detect a valid header.
The APU could maybe parse the data itself, either byte-by-byte during the add() data input, just before add() returns with 'true', or perhaps as a separate 'parse()' method called later, perhaps by some other APU-processing thread.
When reading from a serial port at speed, you typically need some kind of handshaking mechanism to control the flow of data. This can be hardware (e.g. RTS/CTS), software (Xon/Xoff), or controlled by a higher level protocol. If you're reading a large amount of data at speed without handshaking, your UART or serial controller needs to be able to read and buffer all the available data at that speed to ensure no data loss. On 16550 compatible UARTs that you see on Windows PCs, this buffer is just 14 bytes, hence the need for handshaking or a real time OS.
I am using berkeley sockets and TCP (SOCK_STREAM sockets).
The process is:
I connect to a remote address.
I send a message to it.
I receive a message from it.
Imagine I am using the following buffer:
char recv_buffer[3000];
recv(socket, recv_buffer, 3000, 0);
Questions are:
How can I know if after calling recv first time the read buffer is empty or not? If it's not empty I would have to call recv again, but if I do that when it's empty I would have it blocking for much time.
How can I know how many bytes I have readed into recv_buffer? I can't use strlen because the message I receive can contain null bytes.
Thanks.
How can I know if after calling recv
first time the read buffer is empty or
not? If it's not empty I would have to
call recv again, but if I do that when
it's empty I would have it blocking
for much time.
You can use the select or poll system calls along with your socket descriptor to tell if there is data waiting to be read from the socket.
However, usually there should be an agreed-upon protocol that both sender and receiver follow, so that both parties know how much data is to be transferred. For example, perhaps the sender first sends a 2-byte integer indicating the number of bytes it will send. The receiver then first reads this 2-byte integer, so that it knows how many more bytes to read from the socket.
Regardless, as Tony pointed out below, a robust application should use a combination of length-information in the header, combined with polling the socket for additional data before each call to recv, (or using a non-blocking socket). This will prevent your application from blocking in the event that, for example, you know (from the header) that there should still be 100 bytes remaining to read, but the peer fails to send the data for whatever reason (perhaps the peer computer was unexpectedly shut off), thus causing your recv call to block.
How can I know how many bytes I have
readed into recv_buffer? I can't use
strlen because the message I receive
can contain null bytes.
The recv system call will return the number of bytes read, or -1 if an error occurred.
From the man page for recv(2):
[recv] returns the number of bytes
received, or -1 if an error occurred.
The return value will be 0 when the
peer has performed an orderly
shutdown.
How can I know if after calling recv first time the read buffer is empty or not?
Even the first time (after accepting a client), the recv can block and fail if the client connection has been lost. You must either:
use select or poll (BSD sockets) or some OS-specific equivalent, which can tell you whether there is data available on specific socket descriptors (as well as exception conditions, and buffer space you can write more output to)
you can set the socket to be nonblocking, such that recv will only return whatever is immediately available (possibly nothing)
you can create a thread that you can afford to have block recv-ing data, knowing other threads will be doing the other work you're concerned to continue with
How can I know how many bytes I have readed into recv_buffer? I can't use strlen because the message I receive can contain null bytes.
recv() returns the number of bytes read, or -1 on error.
Note that TCP is a byte stream protocol, which means that you're only guaranteed to be able to read and write bytes from it in the correct order, but the message boundaries are not guaranteed to be preserved. So, even if the sender has made a large single write to their socket, it can be fragmented en route and arrive in several smaller blocks, or several smaller send()/write()s can be consolidated and retrieved by one recv()/read().
For that reason, make sure you loop calling recv until you either get all the data you need (i.e. a complete logical message you can process) or an error. You should be prepared/able to handle getting part/all of subsequent sends from your client (if you don't have a protocol where each side only sends after getting a complete message from the other, and are not using headers with message lengths). Note that doing recvs for the message header (with length) then the body can result in a lot more calls to recv(), with a potential adverse affect on performance.
These reliability issues are often ignored. They manifest less often when on a single host, a reliable and fast LAN, with less routers and switches involved, and fewer or non-concurrent messages. Then they may break under load and over more complex networks.
If the recv() returns fewer than 3000 bytes, then you can assume that the read buffer was empty. If it returns 3000 bytes in your 3000 byte buffer, then you'd better know whether to continue. Most protocols include some variation on TLV - type, length, value. Each message contains an indicator of the type of message, some length (possibly implied by the type if the length is fixed), and the value. If, on reading through the data you did receive, you find that the last unit is incomplete, you can assume there is more to be read. You can also make the socket into a non-blocking socket; then the recv() will fail with EAGAIN or EWOULDBLOCK if there is no data read for reading.
The recv() function returns the number of bytes read.
ioctl() with the FIONREAD option tells you how much data can currently be read without blocking.