I want to increase the throughput of my udp gameserver which uses Boost ASIO.
Right now, everytime i need to send a packet, i am putting it in a queue, then checking if there is a pending async_send_to operation, if yes, do nothing, if not, call async_send_to.
Then i wait for the write handler to be called and then call async_send_to for the next packet in queue, if any.
The documentation says that it is the way to do it "for TCP socket", but there is NOTHING on the whole internet about UDP socket.
Try it, search it on stackoverflow, you will see nobody talks about this, and for the 2 questions you will find, the question is left ignored by users.
Why is it kept a secret?
And for the 1million dollar question, can i safely call async_send_to multiple time in a row WITHOUT waiting for the write handler to be called?
Thanks in advance.
This logic is meaningless for the UDP protocol since it doesn't need to block send operation. A datagram is either delivered or lost. UDP don't have to store it in the output buffer and resend indefinitely many times until it get ACK packet.
No, you cannot safely call async_send_to multiple times in a row WITHOUT waiting for the write handler to be called. See Asynchronous IO with Boost.Asio to see precisely why.
However, asio supports scatter gather and so you can call async_send_to with multiple buffers, e.g.:
typedef std::deque<boost::asio::const_buffer> ConstBuffers;
std::string msg_1("Blah");
...
std::string msg_n("Blah");
ConstBuffers buffers;
buffers.push_back(msg_1);
...
buffers.push_back(msg_n);
socket_.async_send_to(buffers, tx_endpoint_, write_handler);
So you could increase your throughput by double buffering your message queue and using gathered writes...
Related
Please consider a simple echo server using TCP and the Winsock DLL. The client application sends messages from multiple threads. The recv call on the server sometimes returns with multiple messages stored in the passed buffer. At this point, there's no chance for the server to know, whether this is one huge message or multiple small messages.
I've read that one could use setsockopt in combination with the TCP_NODELAY option. Besides that MSDN states, that this option is implemented for backward compatibility only, it doesn't even change the behavior described above.
Of course, I could introduce some kind of delimiter at the end of each message and split the message on server-side. But I don't think that's way one should do it. So, what is the right way to do it?
Firstly, TCP_NODELAY was not the right way to do this... TCP is a byte stream protocol and any given connection only maintains the byte ordering - not necessarily the boundaries of any given send/write. It's inherently broken to rely on multiple threads that don't use any synchronisation being able to even keep the messages they want to send together on the stream. For example, say thread 1 wants to send the two-byte message "AB" and thread 2 wants to send "XY"... say thread 1 starts first and the output buffer only has room for one byte, send will enqueue "A" and let thread 1 know it's only sent one byte (so it should loop and retry - preferable after waiting for notification that the output queue has more space). Then, thread 2 might get some or all of "XY" into the queue before thread 1 can get "Y". These sorts of problems become more severe on slower connections, for slow and loaded machines (e.g. perhaps a low-powered phone that's playing video and multitasking while your app runs over 3G).
The ways to ensure the logical messages stay together over TCP include:
have a single sending thread that picks up messages sequentially from a shared queue (a mutex might be used to let the threads enqueue messages)
contest a lock (mutex) so the threads' sends have an uninterrupted ability to loop to send until a complete message is sent (this wouldn't suit some apps because any of the threads could be held up for quite a while doing comms work)
use a separate TCP connection per thread
Hi I'm working on a networking project. I've a socket that is listening incoming data. Now I want to archive this: Socket will receive only 100 packets. And there is 3-4 clients. They are sending random data packets infinitely. I'll receive 100 packets and later I'll process them. After process I'll re-start receiving. But at this time there are some pending send() >> recv() operations. Now I want to cancel/discard pending recv operations. I think we'll recv datas and we'll not process them. Any other suggestions? (sorry for bad question composition)
Shutdown and close the connection. That will cancel everything immediately.
Better yet, rearchitect your application and network protocol so that you can reliably tell how much data to receive.
On Windows you can cancel outstanding receives using CancelIO, but that might result in lost data if the receive just happened to read something.
You can use select() or poll() loops.
you can use signal. recv() will return on receiving a signal so you can send a signal from another task to the task that blocks on recv(). But you need to make sure you don't specify SA_RESTART (see http://pubs.opengroup.org/onlinepubs/9699919799/functions/sigaction.html)
Read http://en.wikipedia.org/wiki/Asynchronous_I/O for more details
I would go with non-blocking sockets + cancellation socket.
You'll have to read into dedicated incremental buffer (as recv() may not receive all the data expected at once - this would be the case if you can only process full messages) and return to select()/poll() in your loop, where you can safely sit and wait for:
next data
next connection
cancellation event from a cancellation socket, to which your other thread will send a cancellation signal (some trivial send()).
UPD: the trivial event may be the number of the socket in the array or its handle - something to identify which one you'd like to cancel.
When i was taking a look at setsockopt from msdn link. i came across a parameter SO_RCVTIMEO, it description is "Sets the timeout, in milliseconds, for blocking receive calls." I thought the socket listen operation is event driven which means when kernel drained frame from NIC card it notify my program socket, so what is the blocking all about?
The recv and WSARecv functions are blocking. They are not event driven (at least not at the calling level). Even when blocking has a timeout (as set with the SO_RECTIMEO option), they are not event driven as far as your code is concerned. In that case, they are just pseudo-blocking (arguably non-blocking depending on how short the timeout is).
When you call WSARecv, it will wait until data is ready to be read. While data is not ready to be read, it just waits. This is why it's considered blocking.
You are correct that at it's core networking is event driven. Under the hood, computers are, by nature, event driven. It's the way hardware works. Hardware interrupts are essentially events. You're right that at a low level what is happening is that your NIC card is telling the OS that it's ready to be read. At that level, it is indeed event based.
The problem is that WSARecv waits for that event.
Here's a hopefully clear analogy. Imagine that you for some reason cannot leave your house. Now imagine that your friend F lives next door. Additionally, assume that your other friend G is at your house.
Now imagine that you give G a piece of paper with a question on it and ask him to take it to F.
Once the question has been sent, imagine that you send G to go get F's response. This is like the recv call. G will wait until F has written down his response, then he will bring it to you. G does not immediately turn around and come back if F hasn't written it yet.
This is where the gap comes from. G is indeed aware of the "F wrote!" events, but you're not. You're not directly watching the piece of paper.
Setting a timeout means that you're telling G to wait at most some amount of time before giving up and coming back. In this situation, G is still waiting on F to write, but if F doesn't write within x milliseconds, G turns around and comes back empty handed.
Basically the pseudo code of recv is vaguely like:
1) is data available?
1a) Yes: read it and return
1b) No: GOTO 2
2) Wait until an event is received
2a) GOTO 1
I know this has been a horribly convoluted explanation, but my main point is this: recv is interacting with the events, not your code. recv blocks until one of those events is received. If a timeout is set, it blocks until either one of those events is received, or the timeout is reached.
Sockets are NOT event-driven by default. You have to write extra code to enable that. A socket is initially created in a blocking mode instead. This means that a call to send(), recv(), or accept() will block the calling thread indefinately by default until the requested operation is finished.
For recv(), that means the calling thread is blocked until there is at least 1 byte available to read from the socket's receive buffer, or until a socket error occurs, whichever occurs first. SO_RCVTIMEO allows you to set a timeout on the blocking read so recv() exits with a WSAETIMEDOUT error if no incoing data becomes available before the timeout elapses.
Another way to implement a timeout is to set the socket to a non-blocking mode instead via ioctlsocket(FIONBIO) and then call select() with a timeout, then call recv() or accept() only if select() reports that the socket is in a readible state, and send() only if select() reports the socket is in a writable state. But this requires more code to manage cases where the socket would enter a blocking state, causing operations to fail with WSAEWOULDBLOCK errors.
how to send and receive data and acknowledgement asynchrounously in c++ socket tcp/ip?, i can see in my program as when send some 145 byte data 50 times , i will be receiving the acknowledgement for the first six as it takes time to give the acknowledgement. I have tried waiting until i receive the acknowledgement for the current sent data, but it takes long time , which i dont want to do. I want to process with the data. So what i thought is i will put receive in a separate thread. and i want to process receive simultaneouly while send is happening? So Asynchronous receive will work? Can anyone help? The class which i am using to receive and send data is a static class. As i needed to call from two other classes to send data to same ip and port.
You can use socket for sending data in one thread and for receiving data in another thread, simultaneously. The thing you cannot do is performing the same action (sending or receiving) on the same socket from different threads.
Nevertheless, always try to avoid adding new threads if unnecessary. You can use some socket functions (select, WSAEventSelect) that will notify you when you can perform read/write on the socket. Put them in a loop in a single thread, wait for events and handle them - read/write in the same thread. If you need to process received data and that takes time, store received data in a queue shared with another, data processing thread. Of course, bear in mind you would need to make this queue thread-safe and synchronise socket and data processing thread.
If on Posix systems, you might need a multiplexing system call like poll or ppoll or pselect or select
If your system is non-Posix (e.g. windows) you should find some similar functionality, or use threads.
I have a socket server, written in C++ using boost::asio, and I'm sending data to a client.
The server sends the data out in chunks, and the client parses each chunk as it receives it. Both are pretty much single threaded right now.
What design should I use on the server to ensure that the server is just writing out the data as fast as it can and never waiting on the client to parse it? I imagine I need to do something asynchronous on the server.
I imagine changes could be made on the client to accomplish this too, but ideally the server should not wait on the client regardless of how the client is written.
I'm writing data to the socket like this:
size_t bytesWritten = m_Socket.Write( boost::asio::buffer(buffer, bufferSize));
Update:
I am going to try using Boost's mechanism to write asynchronously to the socket. See http://www.boost.org/doc/libs/1_36_0/doc/html/boost_asio/tutorial/tutdaytime3/src.html
e.g.
boost::asio::async_write(socket_, boost::asio::buffer(message_),
boost::bind(&tcp_connection::handle_write, shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
Alex
If you set your socket to non-blocking, then writes should fail if they would otherwise block. You can then queue up the data however you like, and arrange for another attempt to be made later to write it. I don't know how to set socket options in the boost socket API, but that's what you're looking for.
But this is probably more trouble than it's worth. You'd need to select a socket that's ready for writing, presumably from several open simultaneously, shove more data into it until it's full, and repeat. I don't know if the boost sockets API has an equivalent of select, so that you can wait on multiple sockets at once until any of them is ready to write.
The reason that servers typically start a thread (or spawn a process) per client connection is precisely so that they can get on with serving other clients while they're waiting on I/O, while avoiding implementing their own queues. The simplest way to "arrange for another attempt later" is just to do blocking I/O in a dedicated thread.
What you can't do, unless boost has done something unusual in its sockets API, is require the OS or the sockets library to queue up arbitrary amounts of data for you without blocking. There may be an async API which will call you back when the data is written.
You can ensure asynchronous communication by transporting the data not over TCP but over UDP.
However, if you need to use TCP, let the client store the data away quickly and process it in a different thread or asynchronously with a cron job.
When you pass data to a socket, it does not wait for the receiver to process it. It does not even wait for the data to be transmitted. The data is put into an outbound queue that is processed by the OS in the background. The writing function returns how many bytes were queued for transmission, not how many bytes were actually transmitted.
Continuing from the comments on Stefan's post:
It is definitely possible to buffer on either the client or server side. But make sure to consider what Neil wrote. If we just begin to buffer data blindly and if the processing can never keep up with the sending then our buffer will grow in a fashion we probably don't want.
Now I recently implemented a straightforward 'NetworkPipe' which was meant to function as a connection between a single client/server, server/client where the outside user doesn't know/care if the Pipe is the client or the server. I implemented a buffering situation similar to what you are asking about, how? Well the class was threaded, this was about the only way I could figure out to cleanly buffer the data. Here is the basic process that I followed, and note that I set a maximum size on the Pipes:
Process 1 starts pipe, defaults to server. Now internal thread waits for client.
Process 2 starts pipe, already a server, defaults to Client.
We are now connected, first thing to do is exchange maximum buffer sizes.
Process 1 writes data (it notes that the other end has an empty buffer [see #3])
Process 2's internal thread (now waiting on a select() for the socket) sees that data is sent and reads it, buffers it. Process 2 now sends back the new buffered size to P1.
So thats a really simplified version but basically by threading it I can always be waiting on a blocking select call, as soon as data arrives I can read and buffer it, I send back the new buffered size. You could do something similar, and buffer the data blindly, its actually quite a bit simpler because you don't have to exchange buffer sizes, but probably a bad idea. So the above example allowed external users to read/write data without blocking their thread (unless the buffer on the other end is full).
I implemented a solution using the boost::asio::async_write method.
Basically:
I have one thread per client (my threads are doing CPU bound work)
As each thread accumulates some amount of data, it writes it to the socket using async_write, not caring if previous writes have completed
The code is careful to manage the lifetime of the socket and the data buffers being written out because the CPU processing finishes before all the data has written out
This works well for me. This enables the server thread to finish as soon as its done its CPU work.
Overall the the time for the client to receive and parse all of its data went down. Similarly the time (clock on the wall time) that the server spends on each client goes down.
Code snippet:
void SocketStream::Write(const char* data, unsigned int dataLength)
{
// Make a copy of the data
// we'll delete it when we get called back via HandleWrite
char* dataCopy = new char[dataLength];
memcpy( dataCopy, data, dataLength );
boost::asio::async_write
(
*m_pSocket,
boost::asio::buffer(dataCopy, dataLength),
boost::bind
(
&SocketStream::HandleWrite, // the address of the method to callback when the write is done
shared_from_this(), // a pointer to this, using shared_from_this to keep us alive
dataCopy, // first parameter to the HandleWrite method
boost::asio::placeholders::error, // placeholder so that async_write can pass us values
boost::asio::placeholders::bytes_transferred
)
);
}
void SocketStream::HandleWrite(const char* data, const boost::system::error_code& error, size_t bytes_transferred)
{
// Deallocate the buffer now that its been written out
delete data;
if ( !error )
{
m_BytesWritten += bytes_transferred;
}
else
{
cout << "SocketStream::HandleWrite received error: " << error.message().c_str() << endl;
}
}