We have an application where we send data through TCP sockets. We use 8 TCP connections for this. The socket send and receive is called in a background thread. There is just one thread which iterates over the array of sockets to send data through all of them (sequential).
The code in sender thread is something like:
for(i = 0; i < 8; i++) {
nBytesWrriten = send (tcpsock[i], data2, nleft, 0))
//error handling and process more data
}
and the receiver thread is like:
for(i = 0; i < 8; i++) {
sz[i] = recv (tcpsock[i], data, MAX_UDT_SIZE, 0);
//process data
}
Everything works fine and the data gets transferred, but sometimes it just takes too long.
On checking logs, I found that in most cases, the sender thread works just fine , but sometimes, there is a huge delay in timestamps(sometimes more than a second) before and after 'send' call.
All of the send and receive action is taking place in a worker thread. Is it something to do with pre-emption of the thread just before/on send call? Can I avoid the pre-emption of the thread just before the send call? Or is it that the receiver thread has not received the data on the socket while send it ready with more data, and therefore it causes the delay?
How do I optimize this as it is taking too long to send data?
Thanks
You should use non-blocking sockets for sending. What might be happening is that one (or more than one) cannot send right away, so it waits until it can send some data, maybe buffers full or whatever.
Using non-blocking sockets it won't stop, but you must check that the data was not sent to some sockets and try again later.
Do a select on each socket before the send to see if you can write without blocking, otherwise you'll block the sends on the other sockets. You'll want to do the same thing on the read side or the lack of readability of one could block available read on others.
Related
I'm doing a program that will permit me to ping a lot of different IPs simultaneously (around 50-70).
I also need to delay the sending of each packet by say 1 ms, for multiple reasons, notably not getting my packets dropped by some routers which drop ICMP packets when there's too much sent at once (which mine does, on the sending machine, not the receiving one).
So I did it in a separate thread a bit like that :
// Send thread
for (;;) {
[...]
for (int i = 0; i < ip_count; i++)
{
// Send() calls sendto()
Send(m_socket, ip_array[i]);
[...]
Sleep(1); // Delay by 1 ms
}
[...]
lock_until_new_send_operation();
}
And in another thread, thread I would like to receive the packets with select(), like that
// Receive thread
FD_ZERO(&m_fdset_read);
FD_SET(m_socket, &m_fdset_read);
int rds_count = select(0, &m_fdset_read, 0, 0, &tvtimeout);
if (rds_count > 0)
ProcessReadySocket(); // Calls recv() and stuff
else {
// Timed out
m_bSendGrapeDone = true;
}
The problem with this approach is that since both calls to select() and sento() use the same non-blocking socket m_socket, calls to sendto() would later block because select() makes sendto() block when both are called simultaneously (for some strange reason... dont see such logic there, since socket is non blocking, but it still does, its ugly).
So I decided to use one socket exclusively for sending then and replaced the line
Send(m_socket, ip_array[i]);
with
Send(m_sendSock, ip_array[i]); // m_sendSock is dedicated to sending only
I read on MSDN that for raw sockets each socket receives all packets for the protocol the socket is set to (mine is IPPROTO_ICMP ofc). Here I quote :
There are further limitations for applications that use a socket of
type SOCK_RAW. For example, all applications listening for a specific
protocol will receive all packets received for this protocol
SO I thought even though my packets are sent with m_sendSock, I could still receive them using select()/recv() on m_socket, turns out I can't, select() never returns the socket is readable. So I'm kind of stuck, I cannot use select() and send() at the same time. Is there something I'm doing wrong?
(by the way I want to use raw sockets, not builtin windows icmp functions)
TL;DR How can I send() and select() simultaneously? Because on the send thread, send() blocks as soon as select() gets called on the receive thread, even if I used FIONBIO on it (non blocking). If I use two different sockets, one for sending and one destined to receive, I receive nothing on the receiving socket...
Thanks!
I am trying to come up how to effectively use ZMQ to multithread (so send doesn't block receive and receive doesn't block send).
I wanted to use ZMQ_DONTWAIT flag but when sending the data, it will sometimes not be send (EAGAIN error, so I would have to re-queue the message which is a waste of resources when dealing with megabytes of data).
I did come up with the following code:
Concurrency::concurrent_queue<zmq::message_t> QUEUE_IN;
Concurrency::concurrent_queue<zmq::message_t> QUEUE_OUT;
void SendThread(zmq::context_t &context) {
zmq::socket_t zmq_socket(context, ZMQ_DEALER);
zmq_socket.connect(string_format("tcp://%s:%s", address, port).c_str());
zmq::message_t reply;
while (true) {
while (QUEUE_OUT.try_pop(reply))
zmq_socket.send(reply);
Sleep(1);
}
}
void RecvThread(zmq::context_t &context) {
zmq::socket_t zmq_socket(context, ZMQ_DEALER);
zmq_socket.connect(string_format("tcp://%s:%s", address, port).c_str());
zmq::message_t reply;
while (true) {
while (zmq_socket.recv(&reply))
QUEUE_IN.push(reply);
}
}
void ConnectionThread()
{
zmq::context_t context(1);
std::thread* threads[2] = {
new std::thread(SendThread, context),
new std::thread(RecvThread, context)
};
threads[0]->join();
}
However that would require two sockets on the server end, and I would need to identify to which I need to send data and to which I need to listen on the server end, right?
Is there no way to use one socket yet use send and receive in a multithreaded environment?
I would maybe like to do it asychroniously on one socket, but after studying the async sample I still don't grasp the idea as there aren't much comments around it.
Avoiding the Sleep
To avoid the sleep, you can use zmq_poll() using a ZMQ_POLLOUT event to protect the send(). You don't need to use ZMQ_DONTWAIT. [I used the C function there, your binding will have the equivalent.]
Routing to RecvThread
One cannot share sockets between threads, so 2 sockets are needed for this to work. The server would only need one socket (presumably ROUTER) that bound to 2 ports. When it receives a message, it will then need to know where to send the reply...
When a ROUTER socket receives a message, the zmq internals adds a frame to the message with the identity of the sender. This frame will be seen by the server code, which would normally use that same identity frame when constructing a message to reply to the sender. In your case, that's the client's SendThread. OTOH, you want to reply to the client's receive socket, so the identity frame must be for that.
The only thing left is how the server obtains the identity frame of the client's receive socket. For that, you'll need to invent a small protocol. Arranging for the client's RecvThread to send one message to the server would almost be enough. The server should understand that message and simply retain the identity frame of the client's receive socket, and use a copy of it when constructing reply messages.
All of this is explained in the guide under "Exploring ROUTER Sockets".
When sending large data (you say you're sending MB of data in a single message), it's going to take some time, ZMQ doesn't "duplex" sending and receiving so that they can both actually happen. The DONTWAIT flag isn't going to help you so much there, its purpose is to ensure that you're not waiting on ZMQ when you could be performing non-ZMQ actions. All messages should still be queued up in any event (barring interference from the High Water Mark)
The only way to safely use multiple threads to parallelize sending and receiving is to use multiple sockets.
But, it's not all bad. If you use one designated send socket and one designated receive socket, then you can use pub/sub, which opens up some interesting options.
Hi I'm working on a networking project. I've a socket that is listening incoming data. Now I want to archive this: Socket will receive only 100 packets. And there is 3-4 clients. They are sending random data packets infinitely. I'll receive 100 packets and later I'll process them. After process I'll re-start receiving. But at this time there are some pending send() >> recv() operations. Now I want to cancel/discard pending recv operations. I think we'll recv datas and we'll not process them. Any other suggestions? (sorry for bad question composition)
Shutdown and close the connection. That will cancel everything immediately.
Better yet, rearchitect your application and network protocol so that you can reliably tell how much data to receive.
On Windows you can cancel outstanding receives using CancelIO, but that might result in lost data if the receive just happened to read something.
You can use select() or poll() loops.
you can use signal. recv() will return on receiving a signal so you can send a signal from another task to the task that blocks on recv(). But you need to make sure you don't specify SA_RESTART (see http://pubs.opengroup.org/onlinepubs/9699919799/functions/sigaction.html)
Read http://en.wikipedia.org/wiki/Asynchronous_I/O for more details
I would go with non-blocking sockets + cancellation socket.
You'll have to read into dedicated incremental buffer (as recv() may not receive all the data expected at once - this would be the case if you can only process full messages) and return to select()/poll() in your loop, where you can safely sit and wait for:
next data
next connection
cancellation event from a cancellation socket, to which your other thread will send a cancellation signal (some trivial send()).
UPD: the trivial event may be the number of the socket in the array or its handle - something to identify which one you'd like to cancel.
I am using Windows socket for my application(winsock2.h). Since the blocking socket doesn't let me control connection timeout, I am using non-blocking one. Right after send command I am using shutdown command to flush(I have to). My timeout is 50ms and the thing I want to know is if the data to be sent is so big, is there a risk of sending only a portion of data or sending nothing at all? Thanks in advance...
hSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
u_long iMode=1;
ioctlsocket(hSocket,FIONBIO,&iMode);
connect(hSocket, (sockaddr*)(&sockAddr),sockAddrSize);
send(hSocket, sendbuf, sendlen, 0);
shutdown(hSocket, SD_BOTH);
Sleep(50);
closesocket(hSocket);
Non-blocking TCP socket and flushing right after send?
There is no such thing as flushing a TCP socket.
Since the blocking socket doesn't let me control connection timeout
False. You can use select() on a blocking socket.
I am using non-blocking one.
Non sequitur.
Right after send command I am using shutdown command to flush(I have to).
You don't have to, and shutdown() doesn't flush anything.
My timeout is 50ms
Why? The time to send data depends on the size of the data. Obviously. It does not make any sense whatsoever to use a fixed timeout for a send.
and the thing I want to know is if the data to be sent is so big, is there a risk of sending only a portion of data or sending nothing at all?
In blocking mode, all the data you provided to send() will be sent if possible. In non-blocking mode, the amount of data represented by the return value of send() will be sent, if possible. In either case the connection will be reset if the send fails. Whatever timeout mechanism you superimpose can't possibly change any of that: specifically, closing the socket asynchronously after a timeout will only cause the close to be appended to the data being sent. It will not cause the send to be aborted.
Your code wouldn't pass any code review known to man. There is zero error checking; the sleep is completely pointless; and shutdown before close is redundant. If the sleep is intended to implement a timeout, it doesn't.
I want to be sending data as fast as possible.
You can't. TCP implements flow control. There is exactly nothing you can do about that. You are rate-limited by the receiver.
Also the 2 possible cases are: server waits too long to accept connection
There is no such case. The client can complete a connection before the server ever calls accept(). If you're trying to implement a connect timeout shorter than the default of about a minute, use select().
or receive.
Nothing you can do about that: see above.
So both connecting and writing should be done in max of 50ms since the time is very important in my situation.
See above. It doesn't make sense to implement a fixed timeout for operations that take variable time. And 50ms is far too short for a connect timeout. If that's a real issue you should keep the connection open so that the connect delay only happens once: in fact you should keep TCP connections open as long as possible anyway.
I have to flush both write and read streams
You can't. There is no operation in TCP that will flush either a read stream or a write stream.
because the server keeps sending me unnecessarly big data and I have limited internet connection.
Another non sequitur. If the server sends you data, you have to read it, otherwise you will stall the server, and that doesn't have anything to do with flushing your own write stream.
Actually I don't even want a single byte from the server
Bad luck. You have to read it. [If you were on BSD Unix you could shutdown the socket for input, which would cause data from the server to be thrown away, but that doesn't work on Windows: it causes the server to get a connection reset.]
Thanks to EJP and Martin, now I have created a second thread to check. Also in the code I posted in my question, I added "counter=0;" line after the "send" line and removed shutdown. It works just as I wanted now. It never waits more than 50ms :) Really big thanks
unsigned __stdcall SecondThreadFunc( void* pArguments )
{
while(1)
{
counter++;
if (counter > 49)
{
closesocket(hSocket);
counter = 0;
printf("\rtimeout");
}
Sleep(1);
}
return 0;
}
I have a socket server, written in C++ using boost::asio, and I'm sending data to a client.
The server sends the data out in chunks, and the client parses each chunk as it receives it. Both are pretty much single threaded right now.
What design should I use on the server to ensure that the server is just writing out the data as fast as it can and never waiting on the client to parse it? I imagine I need to do something asynchronous on the server.
I imagine changes could be made on the client to accomplish this too, but ideally the server should not wait on the client regardless of how the client is written.
I'm writing data to the socket like this:
size_t bytesWritten = m_Socket.Write( boost::asio::buffer(buffer, bufferSize));
Update:
I am going to try using Boost's mechanism to write asynchronously to the socket. See http://www.boost.org/doc/libs/1_36_0/doc/html/boost_asio/tutorial/tutdaytime3/src.html
e.g.
boost::asio::async_write(socket_, boost::asio::buffer(message_),
boost::bind(&tcp_connection::handle_write, shared_from_this(),
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
Alex
If you set your socket to non-blocking, then writes should fail if they would otherwise block. You can then queue up the data however you like, and arrange for another attempt to be made later to write it. I don't know how to set socket options in the boost socket API, but that's what you're looking for.
But this is probably more trouble than it's worth. You'd need to select a socket that's ready for writing, presumably from several open simultaneously, shove more data into it until it's full, and repeat. I don't know if the boost sockets API has an equivalent of select, so that you can wait on multiple sockets at once until any of them is ready to write.
The reason that servers typically start a thread (or spawn a process) per client connection is precisely so that they can get on with serving other clients while they're waiting on I/O, while avoiding implementing their own queues. The simplest way to "arrange for another attempt later" is just to do blocking I/O in a dedicated thread.
What you can't do, unless boost has done something unusual in its sockets API, is require the OS or the sockets library to queue up arbitrary amounts of data for you without blocking. There may be an async API which will call you back when the data is written.
You can ensure asynchronous communication by transporting the data not over TCP but over UDP.
However, if you need to use TCP, let the client store the data away quickly and process it in a different thread or asynchronously with a cron job.
When you pass data to a socket, it does not wait for the receiver to process it. It does not even wait for the data to be transmitted. The data is put into an outbound queue that is processed by the OS in the background. The writing function returns how many bytes were queued for transmission, not how many bytes were actually transmitted.
Continuing from the comments on Stefan's post:
It is definitely possible to buffer on either the client or server side. But make sure to consider what Neil wrote. If we just begin to buffer data blindly and if the processing can never keep up with the sending then our buffer will grow in a fashion we probably don't want.
Now I recently implemented a straightforward 'NetworkPipe' which was meant to function as a connection between a single client/server, server/client where the outside user doesn't know/care if the Pipe is the client or the server. I implemented a buffering situation similar to what you are asking about, how? Well the class was threaded, this was about the only way I could figure out to cleanly buffer the data. Here is the basic process that I followed, and note that I set a maximum size on the Pipes:
Process 1 starts pipe, defaults to server. Now internal thread waits for client.
Process 2 starts pipe, already a server, defaults to Client.
We are now connected, first thing to do is exchange maximum buffer sizes.
Process 1 writes data (it notes that the other end has an empty buffer [see #3])
Process 2's internal thread (now waiting on a select() for the socket) sees that data is sent and reads it, buffers it. Process 2 now sends back the new buffered size to P1.
So thats a really simplified version but basically by threading it I can always be waiting on a blocking select call, as soon as data arrives I can read and buffer it, I send back the new buffered size. You could do something similar, and buffer the data blindly, its actually quite a bit simpler because you don't have to exchange buffer sizes, but probably a bad idea. So the above example allowed external users to read/write data without blocking their thread (unless the buffer on the other end is full).
I implemented a solution using the boost::asio::async_write method.
Basically:
I have one thread per client (my threads are doing CPU bound work)
As each thread accumulates some amount of data, it writes it to the socket using async_write, not caring if previous writes have completed
The code is careful to manage the lifetime of the socket and the data buffers being written out because the CPU processing finishes before all the data has written out
This works well for me. This enables the server thread to finish as soon as its done its CPU work.
Overall the the time for the client to receive and parse all of its data went down. Similarly the time (clock on the wall time) that the server spends on each client goes down.
Code snippet:
void SocketStream::Write(const char* data, unsigned int dataLength)
{
// Make a copy of the data
// we'll delete it when we get called back via HandleWrite
char* dataCopy = new char[dataLength];
memcpy( dataCopy, data, dataLength );
boost::asio::async_write
(
*m_pSocket,
boost::asio::buffer(dataCopy, dataLength),
boost::bind
(
&SocketStream::HandleWrite, // the address of the method to callback when the write is done
shared_from_this(), // a pointer to this, using shared_from_this to keep us alive
dataCopy, // first parameter to the HandleWrite method
boost::asio::placeholders::error, // placeholder so that async_write can pass us values
boost::asio::placeholders::bytes_transferred
)
);
}
void SocketStream::HandleWrite(const char* data, const boost::system::error_code& error, size_t bytes_transferred)
{
// Deallocate the buffer now that its been written out
delete data;
if ( !error )
{
m_BytesWritten += bytes_transferred;
}
else
{
cout << "SocketStream::HandleWrite received error: " << error.message().c_str() << endl;
}
}