I have a pthread running and waiting for messages from a socket. The thread gets blocked to wait a message and doesn't wake up until receiving a new one. Is there a way to send a signal to thread to wake up and for the receive function (recvmsg) to return an error code related to signal?
Yes, by default SIGINT will interrupt all syscalls. From man recv:
EINTR The receive was interrupted by delivery of a signal before any
data were available; see signal(7).
and
MSG_WAITALL (since Linux 2.2)
This flag requests that the operation block until the full request is
satisfied. However, the call may still return less data than requested
if a signal is caught, an error or disconnect occurs, or the next
data to be received is of a different type than that returned.
However, you cannot target a specific thread or specific operation.
If you wish to have this, I suggest using a condition that the receiving thread can explicitely listen for. There is a wellknown trick on linux which allows the receiving thread to use select or poll to listen for the socket and the 'condition' simultaneously[1].
The trick is to open a pipe from the master thread to the client (receiving) thread. The master writes to the pipe upon reaching a certain state (the signal so to speak). The client (receiving) thread can simply poll both the pipe and the socket and only check which of the two awoke it.
[1] normally pthread_cond_wait and poll/select cannot be combined without racing so you'd need to program wait loops with small timeouts. On Win32 by contrast it is as simple as WaitForMultipleObjects and you're done
Related
I am using the poll mechanism to manage upto 100 connections. Is there any standard practice for what the time out value for the poll() call should be or how to determine it.
My case details -
I have one dispatcher thread listening on all the connections. Once a connection becomes read ready, I disable it for polling and forward the connfd to a thread pool processing reads. The dispatcher thread goes back to polling.
The thread pool consumes the read on the connfd and posts it back to the dispatcher so it can add it for polling next. But the dispatcher wouldn't be able to add it for polling until it returns from the poll() call. I need the dispatcher to periodically check if it needs to re-enable polling for any connfd.
What is a good timeout value so the dispatcher thread can periodically stop polling and update its pollfd list.
You don't need to use the timeout (just set it to INF).
Timeout is basically used when an explicit timer operation is needed (some async IO libraries handles this for you).
To wake up a thread sleeping in poll, use the self-pipe trick. On Linux, eventfd is also available for use.
Using timerfd (Linux only), timeout can be completely obsoleted.
I am searching solution to wake-up select call in c++, As per application requirement i cant set timeout because of multiple thread using select system call.
Please see below scenario.
i want to wakeup select system call waiting on other thread. I tried to write data on the thread from main thread but still it is not able to wakeup it.
I want to close thread and socket if there is empty data on this thread.
It is wakes up select call if socket connection is close from other process, but not working with thread.
Does any one have idea regarding this
On a recent Linux you can use eventfd, on everything in general - a pipe, usage - register one side of the pipe in selector for readability along with actual socket(s), to wake up a selector - just write one byte to the other end of the pipe. Alternatively (if your libc has it) you can use pselect with a sigmask to catch the ALRM signal and raise that signal whenever you need to wake the selector up. Be very careful with using signals approach in a multithreaded application (as "I would not use"), as if not done right a signal may be delivered to a random thread.
Thanks all for valuable suggestion, I am able to resolve the issue with shutdown() call on socket FD using reference answer present on this link, it will pass wakeup signal to select, which is waiting for action. We should close socket only after select call otherwise select will not able to get wake up signal.
I have an applications with 2 threads. The first thread (main-thread) and the second thread (tcp-client-thread). The main-thread generates some messages and puts their in queue for tcp-client-thread. tcp-client-thread has to send those messages to server. But, tcp-client-thread also has to receive some messages from server.
How can I do that? recv stops current thread. Set up timeout forrecv? Then after recv timeout check queue (from main-thread) and if there is messages send their is no any messages start recv again?
You can do your I/O in one non-spinning/non-delayed thread but it's much more complex then just simply creating another thread as suggested in another answer. In short, you'll have to modify your code to handle waiting for multiple event types simultaneously, i.e. an event on the socket OR on a condition signalling data to send, for example. On Windows, you'd use something like WSAEventSelect + WaitForMultipleObjects instead of select, and on Linux you'd use something like eventfd with select. Note that when handling the socket, if it's blocking, you'd want to check for readability before issuing a recv and check for writeability before issuing a send so you don't block on one or the other. Like I said though, easier to just create a send thread...
The thing you need is non-blocking/asynchronous I/O.
You should read some theory before trying to forge any code.
This article, for example:
http://www.wangafu.net/~nickm/libevent-book/01_intro.html
If you are going to use 2 threads, you might want to extend to 3 threads. Let the send and receive functions be on separate threads.
The send thread is sleeping until the main thread gives it data. Specifically, a function in the send software unit places data into the queue, then signals the thread to wake up. The thread wakes up and sends data until the queue is empty, then it goes back to sleep.
Conversely, the receive thread sleeps until it receives data. It appends data to another queue, notifies the main thread that data was received and goes back to sleep.
Edit 1: One Thread
Per your title, if you want to perform the I/O in one thread, you will need to have a polling loop (you can have limited waiting, but not advised).
Loop:
if (data received) then place data into input queue.
if (data in input queue) process some data (use small chunks).
if (data in output queue) send some data.
end-loop.
The idea is to keep the blocks of data small to prevent missing of incoming data. The data can be processed and output when there is no data (and with multiple iterations). This will resolve most synchronization issues.
Hi I'm working on a networking project. I've a socket that is listening incoming data. Now I want to archive this: Socket will receive only 100 packets. And there is 3-4 clients. They are sending random data packets infinitely. I'll receive 100 packets and later I'll process them. After process I'll re-start receiving. But at this time there are some pending send() >> recv() operations. Now I want to cancel/discard pending recv operations. I think we'll recv datas and we'll not process them. Any other suggestions? (sorry for bad question composition)
Shutdown and close the connection. That will cancel everything immediately.
Better yet, rearchitect your application and network protocol so that you can reliably tell how much data to receive.
On Windows you can cancel outstanding receives using CancelIO, but that might result in lost data if the receive just happened to read something.
You can use select() or poll() loops.
you can use signal. recv() will return on receiving a signal so you can send a signal from another task to the task that blocks on recv(). But you need to make sure you don't specify SA_RESTART (see http://pubs.opengroup.org/onlinepubs/9699919799/functions/sigaction.html)
Read http://en.wikipedia.org/wiki/Asynchronous_I/O for more details
I would go with non-blocking sockets + cancellation socket.
You'll have to read into dedicated incremental buffer (as recv() may not receive all the data expected at once - this would be the case if you can only process full messages) and return to select()/poll() in your loop, where you can safely sit and wait for:
next data
next connection
cancellation event from a cancellation socket, to which your other thread will send a cancellation signal (some trivial send()).
UPD: the trivial event may be the number of the socket in the array or its handle - something to identify which one you'd like to cancel.
In my program there is one thread (receiving thread) that is responsible for receiving requests from a TCP socket and there are many threads (worker threads) that are responsible for processing the received requests. Once a request is processed I need to send an answer over TCP.
And here is a question. I would like to send TCP data in the same thread that I use for receiving data. This thread after receiving data usually waits for new data in select(). So once a worker thread finished processing a request and put an answer in the output queue it has to signal the receiving thread that there are data to send. The problem is that I don't know how to cancel waiting in select() in order to get out of waiting and to call send() .
Or shall I use another thread solely for sending data over TCP?
Updated
MSalters, Artyom thank you for you answers!
MSalters, having read your answer I found this site: Winsock 2 I/O Methods and read about WSAWaitForMultipleEvents(). My program in fact must work both on HP-UX and Windows I finally decided to use the approach that had been suggested by Artyom.
You need to use something similar to safe-pipe trick, but in your case you need to use a pair of connected TCP sockets.
Create a pair of sockets.
Add one to the select and wait on it as well
Notify by writing to other socket from other threads.
Select is immediately waken-up as one of the sockets is readable, reads all the
data in this special socket and check all data in queues to send/recv
How to create pair of sockets under Windows?
inline void pair(SOCKET fds[2])
{
struct sockaddr_in inaddr;
struct sockaddr addr;
SOCKET lst=::socket(AF_INET, SOCK_STREAM,IPPROTO_TCP);
memset(&inaddr, 0, sizeof(inaddr));
memset(&addr, 0, sizeof(addr));
inaddr.sin_family = AF_INET;
inaddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
inaddr.sin_port = 0;
int yes=1;
setsockopt(lst,SOL_SOCKET,SO_REUSEADDR,(char*)&yes,sizeof(yes));
bind(lst,(struct sockaddr *)&inaddr,sizeof(inaddr));
listen(lst,1);
int len=sizeof(inaddr);
getsockname(lst, &addr,&len);
fds[0]=::socket(AF_INET, SOCK_STREAM,0);
connect(fds[0],&addr,len);
fds[1]=accept(lst,0,0);
closesocket(lst);
}
Of course some checks should be added for return values.
select is not the native API for Windows. The native way is WSAWaitForMultipleEvents. If you use this to create an alertable wait, you can use QueueUserAPC to instruct the waiting thread to send data. (This might also mean you don't have to implement your own output queue)
See also this post:
How to signal select() to return immediately?
For unix, use an anonymous pipe. For Windows:
Unblocking can be achieved by adding a dummy (unbound) datagram socket to fd_set and then closing it. To make this thread safe, use QueueUserAPC:
The only way I found to make this multi-threadsafe is to close and recreate the socket in the same thread as the select statement is running. Of course this is difficult if the thread is blocking on the select. And then comes in the windows call QueueUserAPC. When windows is blocking in the select statement, the thread can handle Asynchronous Procedure Calls. You can schedule this from a different thread using QueueUserAPC. Windows interrupts the select, executes your function in the same thread, and continues with the select statement. You can now in your APC method close the socket and recreate it. Guaranteed thread safe and you will never loose a signal.
The typical model is for the worker to handle its own writing. Is there a reason why you want to send all the output-IO through selecting thread?
If you're sure of this model, you could have your workers send data back to the master thread using file descriptors as well (pipe(2)) and simply add those descriptors to your select() call.
And, if you're especially sure that you're not going to use pipes to send data back to your master process, the select call allows you to specify a timeout. You can busy-wait while checking your worker threads, and periodically call select to figure out which TCP sockets to read from.
Another quick&dirty solution is to add localhost sockets to the set. Now use those sockets as the inter-thread communication queues. Each worker thread simply sends something to its socket, which ends up on the corresponding socket in your receiving thread. This wakes up the select(), and your receiving thread can then echo the message on the appropriate outgoing socket.