What is the best way to handle an interrupt signal in infinite loop in server application?
I develop simple FTP Server where every client has its own thread. And now, what I want to do is to handle interrupt signal and then interrupt every thread in the vector of client threads.
In every thread I want to have an opportunity to send to client some response while sending/receiving some file and manually close socket. In main thread I want to only log that server was aborted and close server socket.
I had an idea to implement my server class FTPServer like singleton and in signal() call a function which calls method abort() of FTPServer instance. But I don't know what it will do when the main infinite loop is still running and accept() is still waiting for a new client ... And this pattern I can not use in client threads because there are not only one instance ..
My methods are:
void FTPServer::run() {
while ( 1 ){
int cliFd = TCPController::acceptClient ( m_serverFD );
serveNewClient(cliFd);
}
}
void FTPServer::serveNewClient( int clientFD ){
m_client_threads.push_back( thread(&FTPServer::clientThread, *this, clientFD) );
}
void FTPServer::clientThread( int clientFD ){
ClientThread client(clientFD);
client.run();
}
So I want to handle interrupt signal, break(?) the main infinite loop and call abort() method.
I was searching for something like this:
void FTPServer::run() {
try{
while ( 1 ){
int cliFd = TCPController::acceptClient ( m_serverFD );
serveNewClient(cliFd);
}
catch( RuntimeException & e ){
this.abort()
}
}
But I didn't find anything .. :(
In every client thread I have infinite loop too - it waits for client's commands ...
So, please, can you tell me, what is the best way to handle it?
Thank you !
You have multiple threads, each with its own loop. What you should do is make sure your thread loops frequently check to see if they should be aborted. So instead of blocking indefinitely on acceptClient, accept a connection with a timeout. When the timeout expires, or when a connection is accepted, check if the thread should quit.
Related
im trying to connect to a server via boost asio and beast. I need to send heartbeats to the server every 40 seconds, but when I try to, my write requests get stuck in a queue and never get executed, unless the server sends something first.
I have this code to look for new messages that come in.
this->ioContext.run();
thread heartbeatThread(&client::heartbeatCycle, this);
while (this->p->is_socket_open()) {
this->ioContext.restart();
this->p->asyncQueue("", true);
this->ioContext.run();
}
The asyncQueue function just calls async_read, and blocks the io context. The heartbeatCycle tries to send heartbeats, but gets stuck in the queue. If I force it to send anyways, I get
Assertion failed: (id_ != T::id), function try_lock, file soft_mutex.hpp, line 89.
When the server sends a message, the queue is unblocked, and all the queued messages go through, until there is no more work, and the io_context starts blocking again.
So my main question is, is there any way to unblock the io context without having the server send a message? If not, is there a way to emulate the server sending a message?
Thanks!
EDIT:
I have this queue function that queues messages being sent called asyncQueue.
void session::asyncQueue(const string& payload, const bool& madeAfterLoop)
{
if(!payload.empty())
{
queue_.emplace_back(payload);
}
if(payload.empty() && madeAfterLoop)
{
queue_.emplace_back("KEEPALIVE");
}
// If there is something to write, write it.
if(!currentlyQueued_ && !queue_.empty() && queue_.at(0) != "KEEPALIVE")
{
currentlyQueued_ = true;
ws_.async_write(
net::buffer(queue_.at(0)),
beast::bind_front_handler(
&session::on_write,
shared_from_this()));
queue_.erase(queue_.begin());
}
// If there is nothing to write, read the buffer to keep stream alive
if(!currentlyQueued_ && !queue_.empty())
{
currentlyQueued_ = true;
ws_.async_read(
buffer_,
beast::bind_front_handler(
&session::on_read,
shared_from_this()));
queue_.erase(queue_.begin());
}
}
The problem is when the code has nothing no work left to do, it calls async read, and gets stuck until the server sends something.
In the function where I initialized the io_context, I also created a separate thread to send heartbeats every x seconds.
void client::heartbeatCycle()
{
while(this->p->is_socket_open())
{
this->p->asyncQueue(bot::websocket::sendEvents::getHeartbeatEvent(cache_), true );
this_thread::sleep_for(chrono::milliseconds(10000));
}
}
Lastly, I have these 2 lines in my on_read function that runs whenever async read is called.
currentlyQueued_ = false;
asyncQueue();
Once there is no more work to do, the program calls async_read but currentlyQueued_ is never set to false.
The problem is the io_context is stuck looking for something to read. What can I do to stop the io_context from blocking the heartbeats from sending?
The only thing I have found that stops the io_context from blocking is when the server sends me a message. When it does, currentlyQueued_ is set to false, and the queue able to run and the queue is cleared.
That is the reason im looking for something that can emulate the server sending me a message. So is there a function that can do that in asio/beast? Or am I going about this the wrong way.
Thanks so much for your help.
The idea is to run the io_service elsewhere (on a thread, or in main, after starting an async chain).
Right now you're calling restart() on it which simply doesn't afford continuous operation. Why stop() or let it run out of work at all?
Note, manually starting threads is atypical and unsafe.
I would give examples, but lots already exist (also on this site). I'd need to see question code with more detail to give concrete suggestions.
I have a UDP multicast receiver that will start an std::thread then detach() it to handle the receiving of messages, and it should be able to interrupt the blocking call when called upon.
void startThread()
{
if(flagIsNotset)
{
//set flag
std::thread t(&receiveData, this, socketDetails);
t.detach();
}
}
void receiveData(socketDetails)
{
//initilaise socket
//bind socket to interface
//binds multicast address & port to interface
while(true)
{
//recvfrom(...)
//handle received data
}
//prints a message to know that the thread has exited the while loop
//unset flag
}
void interrupt()
{
//unset flag is shifted here after realizing it did not reach the end of receiveData() when closesocket was called for the first time.
closesocket(s);
}
everything was fine, I can receive UDP messages etc. but when interrupt is first called: it does not reach the message printing part & setting flag portion. when I shift the //set flag codes to interrupt(), I can confirm that for the first time (of calling interrupt() then startThread()), the message is not printed but on subsequent calls, it reached the end of the receiveData function as intended.
My question are as follows :
1) What happened to the first thread that was detached? is it stuck in recvfrom forever and was not interrupted by closesocket() ?
2) Will this cause memory leaking ? that is my primary concern...as I am unable to figure out the state of the first thread that was detached, will setting the thread t as a private member and calling the t::~thread() in interrupt() helps to ensure that the thread is terminated when interrupt is called?
3) Repeating startThread() and interrupt() does not seems to create a duplicate thread that is listening from the same socket. (i.e always receive one copy of the multicast message)
4) If there is any suggestions to improve this code, please feel free to comment.
I'm doing some event handling with C++ and pthreads. I have a main thread that reads from event queue I defined, and a worker thread that fills the event queue. The queue is of course thread safe.
The worker thread have a list of file descriptors and create an epoll system call to get events on those file descriptors. It uses epoll_wait to wait for events on the fd's.
Now the problem. Assuming I want to terminate my application cleanly, how can I cancel the worker thread properly? epoll_wait is not one of the cancellation points of pthread(7) so it cannot react properly on pthread_cancel.
The worker thread main() looks like this
while(m_WorkerRunning) {
epoll_wait(m_EpollDescriptor, events, MAXEVENTS, -1);
//handle events and insert to queue
}
The m_WorkerRunning is set to true when the thread starts and it looks like I can interrupt the thread by settings m_WorkerRunning to false from the main thread. The problem is that epoll_wait theoretically can wait forever.
Other solution I though about is: instead of waiting forever (-1) I can wait for example X time slots, then handle properly no-events case and if m_WorkerRunning == false then exit the loop and terminate the worker thread cleanly. The main thread then sets m_WorkerRunning to false, and sleeps X. However I'm not sure about the performance of such epoll_wait and also not sure what would be the correct X? 500ms? 1s? 10s?
I'd like to hear some experienced advises!
More relevant information: the fd's I'm waiting events on, are devices in /dev/input so technically I'm doing some sort of input subsystem. The targeted OS is Linux (latest kernel) on ARM architecture.
Thanks!
alk's answer above is almost correct. The difference, however, is very dangerous.
If you are going to send a signal in order to wake up epoll_wait, never use epoll_wait. You must use epoll_pwait, or you might run into a race with your epoll never waking up.
Signals arrive asynchronously. If your SIGUSR1 arrives after you've checked your shutdown procedure, but before your loop returns to the epoll_wait, then the signal will not interrupt the wait (as there is none), but neither will the program exit.
This might be very likely or extremely unlikely, depending on how long the loop takes in relation to how much time is spent in the wait, but it is a bug one way or the other.
Another problem with alk's answer is that it does not check why the wait was interrupted. It might be any number of reasons, some unrelated to your exit.
For more information, see the man page for pselect. epoll_pwait works in a similar way.
Also, never send signals to threads using kill. Use pthread_kill instead. kill's behavior when sending signals is, at best, undefined. There is no guarantee that the correct thread will receive it, which might cause an unrelated system call to be interrupted, or nothing at all to happen.
You could send the thread a signal which would interupt the blocking call to epoll_wait(). If doing so modify your code like this:
while(m_WorkerRunning)
{
int result = epoll_wait(m_EpollDescriptor, events, MAXEVENTS, -1);
if (-1 == result)
{
if (EINTR == errno)
{
/* Handle shutdown request here. */
break;
}
else
{
/* Error handling goes here. */
}
}
/* Handle events and insert to queue. */
}
A way to add a signal handler:
#include <signal.h>
/* A generic signal handler doing nothing */
void signal_handler(int sig)
{
sig = sig; /* Cheat compiler to not give a warning about an unused variable. */
}
/* Wrapper to set a signal handler */
int signal_handler_set(int sig, void (*sa_handler)(int))
{
struct sigaction sa = {0};
sa.sa_handler = sa_handler;
return sigaction(sig, &sa, NULL);
}
To set this handler for the signal SIGUSR1 do:
if (-1 == signal_handler_set(SIGUSR1, signal_handler))
{
perror("signal_handler_set() failed");
}
To send a signal SIGUSR1 from another process:
if (-1 == kill(<target process' pid>, SIGUSR1))
{
perror("kill() failed");
}
To have a process send a signal to itself:
if (-1 == raise(SIGUSR1))
{
perror("raise() failed");
}
When doing socket programming, with multi-threading,
if a thread is blocked on Accept Function,
and main thread is trying to shut down the process,
how to break the accept function in order to pthread_join safely?
I have vague memory of how to do this by connection itself to its own port in order to break the accept function.
Any solution will be thankful.
Cheers
Some choices:
a) Use non-blocking
b) Use AcceptEx() to wait on an extra signal, (Windows)
c) Close the listening socket from another thread to make Accept() return with an error/exception.
d) Open a temporary local connection from another thread to make Accept() return with the temp connection
The typical approach to this is not to use accept() unless there is something to accept! The way to do this is to poll() the corresponding socket with a suitable time-out in a loop. The loop checks if it is meant to exit because a suitably synchronized flag was set.
An alternative is to send the blocked thread a signal, e.g., using pthread_kill(). This gets out of the blocked accept() with a suitable error indication. Again, the next step is to check some flag to see if the thread is meant to exit. My preference is the first approach, though.
Depending on your system, if it is available, I would use a select function to wait for the server socket to have a read, indicating a socket is trying to connect. The amount of time to time to wait for a connection can be set/adjusted to to what every time you want to wait for a client to connect(infinity, to seconds, to 0 which will just check and return). The return status needs to be checked to see if the time limit was reached (no socket is trying to connect), or if there is something waiting to be serviced (your server socket indicating there is a client which would like to connect). You can then execute the accept knowing there is a socket to connect based on the returned status.
If available I would use a select function with a timeout in a loop to achieve this functionality.
as Glenn suggested
The select function with a timeout value will wait for a socket to connect for a set period of time. If a socket attempts to connect it can be accepted during that period. By looping this select with a timeout it is possible to check for new connections until the break condition is met.
Here is an example:
std::atomic<bool> stopThread;
void theThread ( std::atomic<bool> & quit )
{
struct timeval tv;
int activity;
...
while(!quit)
{
// reset the time value for select timeout
tv.tv_sec = 0;
tv.tv_usec = 1000000;
...
//wait for an activity on one of the sockets
activity = select( max_sd + 1 , &readfds , NULL , NULL , &tv);
if ((activity < 0) && (errno!=EINTR))
{
printf("select error");
}
if (FD_ISSET(master_socket, &readfds))
{
if ((new_socket = accept(master_socket, (struct sockaddr *)&address, (socklen_t*)&addrlen))<0)
{
perror("accept");
exit(EXIT_FAILURE);
}
...
}
}
int main(int argc, char** argv)
{
...
stopThread = false;
std::thread foo(theThread, std::ref(stopThread));
...
stopThread = true;
foo.join();
return 0;
}
A more complete example of 'Select' http://www.binarytides.com
I am pretty new to C++ so I am sure my code and answer can be improved.
Sounds like what you are looking for is this: You set a special flag variable known to the listening/accepting socket, and then let the main thread open a connection to the listening/accepting socket. The listening/accepting socket/thread has to check the flag every time it accepts a connection in order to know when to shut down.
Typically if you want to do multi-threaded networking, you would spawn a thread once a connection is made (or ready to be made). If you want to lower the overhead, a thread pool isn't too hard to implement.
I have an ACE reactor that accepts socket connections and listens for the incoming data on those connections. The reactor runs in a dedicated thread. This is the thread's entry function:
int TcpServer::svc()
{
LogDebug("The TCP server on %i is running", mLocalAddr.get_port_number());
// The current thread will own the reactor. By default, a reactor is owned by
// the creating thread. A reactor cannot run from not owning thread.
if (mReactor.owner(ACE_Thread::self()) != 0)
{
LogThrow("Could not change the owner of the reactor");
}
if (mReactor.run_reactor_event_loop() != 0)
{
LogWarning("Reactor loop has quit with an error.");
}
return 0;
}
Once in a while run_reactor_event_loop exits with -1 and errno reports that the reason is "interrupted system call". How can I handle the situation? From what I know I have two options: call run_reactor_event_loop again or configure the interrupted call to be called again using sigaction and SA_RESTART.
Is it safe to call run_reactor_event_loop again?
What does ACE_Reactor::restart method do? It looks like it is supposed to restart the loop? Will it help?
How safe it to turn on SA_RESTART? Does it mean, for example, that ^C won't stop my application?
Are there any other ways to handle the situation?
Check how Reactor is constructed. ACE_Reactor::open() cal, takes "restart" parameter (default = false) that tells it to restart handle_events method automatically after interruption.