I have a client process that forks a child process to listen for incoming RPCs via the svc_run() method. What I need to do is kill of that child process from the parent and then re-fork the child process providing it a new CLIENT* to a new RPC server.
Here is the bits of my code that are relevant:
// Client Main
CLIENT* connectionToServer;
int pipe[2];
int childPID;
int parentPID;
static void usr2Signal()
{
ServerData sd;
clnt_destroy(connectionToServer);
(void) read(pipe[0], &sd, sizeof(sd));
// Kill child process.
kill(childPID, SIGTERM);
close(pipe[0]);
// RPC connection to the new server
CLIENT *newServerConn =
clnt_create(
sd.ip,
sd.programNum,
1,
"tcp");
if (!newServerConn)
{
// Connection error.
exit(1);
}
connectionToServer = newServerConn;
// Respawn child process.
if (pipe(pipe) == -1)
{
// Pipe error.
exit(2);
}
childPID = fork();
if (childPID == -1)
{
// Fork error.
exit(3);
}
if (childPID == 0)
{
// child closes read pipe and listens for RPCs.
close(pipe[0]);
parentPID = getppid();
svc_run();
}
else
{
// parent closes write pipe and returns to event loop.
close(pipe[1]);
}
}
int main(int argc, char *argv[])
{
/* Some initialization code */
transp = svctcp_create(RPC_ANYSOCK, 0, 0);
if (transp == NULL) {
// TCP connection error.
exit(1);
}
if (!svc_register(transp, /*other RPC program args*/, IPPROTO_TCP))
{
// RPC register error
exit(1);
}
connectionToServer = clnt_create(
192.168.x.xxx, // Server IP.
0x20000123, // Server RPC Program Number
1, // RPC Version
"tcp");
if (!connectionToServer)
{
// Connection error
exit(1);
}
// Spawn child process first time.
if (pipe(pipe) == -1)
{
// Pipe error
exit(1);
}
childPID = fork();
if (childPID == -1)
{
// Fork error.
exit(1);
}
if (childPID == 0)
{
// Close child's read pipe.
close(pipe[0]);
parentPID = getppid();
// Listen for incoming RPCs.
svc_run ();
exit (1);
}
/* Signal/Communication Code */
// Close parent write pipe.
close(pipe[1]);
// Parent runs in event loop infinitely until a signal is sent.
eventLoop();
cleanup();
}
In my server code I have service call that initiates the new connection. This call is invoked by some other operation on the server.
// Server Services
void newserverconnection_1_svc(int *unused, struct svc_req *s)
{
// This service is defined in the server code
ServerData sd;
/* Fill sd with data:
Target IP: 192.168.a.aaa
RPC Program Number: 0x20000321
... other data
*/
connecttonewserver_1(&sd, connectionToServer); // A client service.
}
Back in my client I have the following service:
// Client Service
void connecttonewserver_1_svc(ServerData *sd, struct svc_req *s)
{
// Send the new server connection data to the parent client processs
// via the pipe and signal the parent.
write(pipe[1], sd, sizeof(sd));
kill(parentPID, SIGUSR2);
}
My problem is, everything runs fine until I initiate the new connection. I do not get into any of my error sections, but about 5 seconds after setting up the new connection, my client becomes unresponsive. It does not crash and the child process seems to still be alive also, but my client will no longer receive RPCs or show any print statements when my events defined in the event loop for the parent are triggered by mouse clicks. I am probably doing something slightly wrong to spawn this new RPC loop for the child process, but I can't see what. Any ideas?
So this solution achieves the result I was looking for, but is definitely far from perfect.
static void usr2Signal()
{
ServerData sd;
// clnt_destroy(connectionToServer); // Removed this as it closes the RPC connection.
(void) read(pipe[0], &sd, sizeof(sd));
// Removed these. Killing the child process also seems to close the
// connection. Just let the child run.
// kill(childPID, SIGTERM);
// close(pipe[0]);
// RPC connection to the new server
CLIENT *newServerConn =
clnt_create(
sd.ip,
sd.programNum,
1,
"tcp");
if (!newServerConn)
{
// Connection error.
exit(1);
}
// This is the only necessary line. Note that the old
// connectionToServer pointer was not deregistered/deallocated,
// so this causes a memory leak, but is a quick fix to my issue.
connectionToServer = newServerConn;
// Removed the rest of the code that spawns a new child process
// as it is not needed anymore.
}
I have made my first linux service with C++.
pid_t pid, sid;
pid = fork();
if (pid < 0) {
exit(EXIT_FAILURE);
}
if (pid>0) {
exit(EXIT_SUCCESS);
}
umask(0);
sid = setsid();
if (sid < 0) {
exit(EXIT_FAILURE);
}
if ((chdir("/")) < 0) {
exit(EXIT_FAILURE);
}
close(STDIN_FILENO);
close(STDOUT_FILENO);
close(STDERR_FILENO);
while (1) {
????????
//sleep(10);
}
exit(EXIT_SUCCESS);
What it would do is to wait for my signal and when it receives it to do some tasks and then again wait for my next signal.
I would send my signal (or whatever) somehow from within my c++ app that runs on same machine. Seems like a mechanism of semaphore between two apps. But in this case one is a linux service, and I do not know how the service could wait my signal.
How could I achieve this? What are my alternatives?
Thanks.
Note: The word "signal" caused to confusion. I didn't intend to use that word as technically. I just mean that I need to talk to my linux service from within my cpp app.
NOTE 2: Using signal is not useful because in its handler almost doing any thing is unsafe, whereas I need to do lots of things. (I dont know if I could start a thread, at least!)
Here is an example of an handler that takes care of SIGHUP and SIGTERM, your program could send these signals using kill -9 processid or kill -HUP processid of course there is a few other signals you could use for this purpose check man signal
void handler (int signal_number){
//action
exit(1);
}
And in the main program
struct sigaction act;
struct sigaction act2;
memset (&act, 0, sizeof (act));
memset (&act2, 0, sizeof (act2));
act.sa_handler = handler;
act2.sa_handler = handler;
if (sigaction (SIGHUP, &act, NULL) < 0) {
perror ("sigaction");
}
if (sigaction (SIGTERM, &act, NULL) < 0) {
perror ("sigaction");
}
//wait here for ever or do something.
Finally I have found the right keywords to google what I needed to know.
Here are the alternative ways to communicate between different processes:
http://www.tldp.org/LDP/lpg/node7.html
I'm developing a server-client project based on Winsock in c++. I have designed the server and the client sides so that they can send and receive text messages and files as well.
Then I decided to go for audio communication between server and client. I've actually implemented that however I've figured it out that I've done everything using TCP protocol and that for the audio communication it is better to use UDP protocol.
Then I've searched over the internet and found out that it is possible to use both TCP and UDP alongside each other.
I've tried to use the UDP protocol but I didn't have any major progresses.
My problem is I use both recv() and recvFrom() in a while loop like this:
while (true)
{
buflen = recv(clientS, buffer, 1024, NULL);
if (buflen > 0)
{
// Send the received buffer
}
else if (buflen == 0)
{
printf("closed\n");
break;
}
buflen = recvfrom(udpS, buffer, 1024, NULL, (struct sockaddr*)&_s, &_size);
But the recvFrom() blocks. I think I haven't done the job properly but I couldn't find out how to do it.
Here Server in C accepting UDP and TCP connections I found a similar question but the answers were just explanations and there were no sample codes to demonstrate the point clearly.
Now I need you to help me undestand clearly how to receive data from both TCP and UPD connections.
Any help is appreciated.
When dealing with multiple sockets at a time, use select() to know which socket has data pending before you read it, eg:
while (true)
{
fd_set rfd;
FD_ZERO(&rfd);
FD_SET(clientS, &rfd);
FD_SET(udpS, &rfd);
struct timeval timeout;
timeout.tv_sec = ...;
timeout.tv_usec = ...;
int ret = select(0, &rfd, NULL, NULL, &timeout);
if (ret == SOCKET_ERROR)
{
// handle error
break;
}
if (ret == 0)
{
// handle timeout
continue;
}
// at least one socket is readable, figure out which one(s)...
if (FD_ISSET(clientS, &rfd))
{
buflen = recv(clientS, buffer, 1024, NULL);
if (buflen == SOCKET_ERROR)
{
// handle error...
printf("error\n");
}
else if (buflen == 0)
{
// handle disconnect...
printf("closed\n");
}
else
{
// handle received data...
}
}
if (FD_ISSET(udpS, &rfd))
{
buflen = recvfrom(udpS, buffer, 1024, NULL, (struct sockaddr*)&_s, &_size);
//...
}
}
I had issues trying to background a command in my app, so I was told here to double fork and clear some of the settings, so this was my result:
if((pid = fork()) < 0)
perror("Error with Fork()");
else if(pid > 0) {
return "";
}
if (setsid()==-1) {
Log("failed to become a session leader");
}
if (chdir("/") == -1) {
Log("failed to change working directory");
}
umask(0);
close(STDIN_FILENO);
close(STDOUT_FILENO);
close(STDERR_FILENO);
if (open("/dev/null",O_RDONLY) == -1) {
Log("failed to reopen stdin");
}
if (open("/dev/null",O_WRONLY) == -1) {
Log("failed to reopen stdout");
}
if (open("/dev/null",O_RDWR) == -1) {
Log("failed to reopen stderr");
}
signal(SIGHUP, SIG_IGN);
Log("No return, forking..");
if((pid = fork()) < 0)
perror("Error with Fork()");
else if(pid > 0) {
return "";
} else {
if(execl("/bin/bash", "/bin/bash", "-c", cmddo, (char*) 0) < 0) perror("execl()");
exit(0);
}
Double forking fixed the issue of the execl'd proc stopping when its parent is closed but it left me with the execl'd process holding onto the parents socket, so when the parent tries to start again it can't.
Here is my parent socket stuff:
ServerSocket server(listenport);
while(true)
{
ServerSocket* new_sock = new ServerSocket();
server.accept (*new_sock);
pthread_t thread;
int rc = pthread_create(&thread, NULL, &LoadThread, (void*)(new_sock));
if (rc) Log_warn("Fatal Error: pthread_create() #%d", rc);
pthread_detach(thread);
}
I was told to FD_CLOEXEC on the socket in my last question but I do not understand how-to do that - and google (plus stack) isn't showing me much help in regards to that.
How do I FD_CLOEXEC on my ServerSocket() so when I fork/execl a sub process it won't hang my socket?
Thanks :D
ANSWER:
As told below to clear the fd's out - my code actually had this, and it worked for me:
struct rlimit rl;
int i;
if (rl.rlim_max == RLIM_INFINITY)
rl.rlim_max = 1024;
for (i = 0; (unsigned) i < rl.rlim_max; i++)
close(i);
FD_CLOEXEC is a flag that can be set on the file descriptor -- its effect is that when a process holding the handle calls exec(), the descriptor is closed.
Use
fcntl(fd, F_SETFD, (long)FD_CLOEXEC);
to set the flag; for this to work, you need to access the actual file descriptor.
Also, setsid() is fully sufficient to disassociate yourself from the parent process group, and while double fork() also works, it does not earn you brownie points with the embedded folks.
And last, there is no guarantee that after closing the first three filedescriptors, the next fds opened will be those three; it is better to use
fd newstdin = open(...);
if(dup2(newstdin, STDIN_FILENO) != 0) { /* handle error */ }
close(newstdin);
Imagine I have a process that starts several child processes. The parent needs to know when a child exits.
I can use waitpid, but then if/when the parent needs to exit I have no way of telling the thread that is blocked in waitpid to exit gracefully and join it. It's nice to have things clean up themselves, but it may not be that big of a deal.
I can use waitpid with WNOHANG, and then sleep for some arbitrary time to prevent a busy wait. However then I can only know if a child has exited every so often. In my case it may not be super critical that I know when a child exits right away, but I'd like to know ASAP...
I can use a signal handler for SIGCHLD, and in the signal handler do whatever I was going to do when a child exits, or send a message to a different thread to do some action. But using a signal handler obfuscates the flow of the code a little bit.
What I'd really like to do is use waitpid on some timeout, say 5 sec. Since exiting the process isn't a time critical operation, I can lazily signal the thread to exit, while still having it blocked in waitpid the rest of the time, always ready to react. Is there such a call in linux? Of the alternatives, which one is best?
EDIT:
Another method based on the replies would be to block SIGCHLD in all threads with pthread \ _sigmask(). Then in one thread, keep calling sigtimedwait() while looking for SIGCHLD. This means that I can time out on that call and check whether the thread should exit, and if not, remain blocked waiting for the signal. Once a SIGCHLD is delivered to this thread, we can react to it immediately, and in line of the wait thread, without using a signal handler.
Don't mix alarm() with wait(). You can lose error information that way.
Use the self-pipe trick. This turns any signal into a select()able event:
int selfpipe[2];
void selfpipe_sigh(int n)
{
int save_errno = errno;
(void)write(selfpipe[1], "",1);
errno = save_errno;
}
void selfpipe_setup(void)
{
static struct sigaction act;
if (pipe(selfpipe) == -1) { abort(); }
fcntl(selfpipe[0],F_SETFL,fcntl(selfpipe[0],F_GETFL)|O_NONBLOCK);
fcntl(selfpipe[1],F_SETFL,fcntl(selfpipe[1],F_GETFL)|O_NONBLOCK);
memset(&act, 0, sizeof(act));
act.sa_handler = selfpipe_sigh;
sigaction(SIGCHLD, &act, NULL);
}
Then, your waitpid-like function looks like this:
int selfpipe_waitpid(void)
{
static char dummy[4096];
fd_set rfds;
struct timeval tv;
int died = 0, st;
tv.tv_sec = 5;
tv.tv_usec = 0;
FD_ZERO(&rfds);
FD_SET(selfpipe[0], &rfds);
if (select(selfpipe[0]+1, &rfds, NULL, NULL, &tv) > 0) {
while (read(selfpipe[0],dummy,sizeof(dummy)) > 0);
while (waitpid(-1, &st, WNOHANG) != -1) died++;
}
return died;
}
You can see in selfpipe_waitpid() how you can control the timeout and even mix with other select()-based IO.
Fork an intermediate child, which forks the real child and a timeout process and waits for all (both) of its children. When one exits, it'll kill the other one and exit.
pid_t intermediate_pid = fork();
if (intermediate_pid == 0) {
pid_t worker_pid = fork();
if (worker_pid == 0) {
do_work();
_exit(0);
}
pid_t timeout_pid = fork();
if (timeout_pid == 0) {
sleep(timeout_time);
_exit(0);
}
pid_t exited_pid = wait(NULL);
if (exited_pid == worker_pid) {
kill(timeout_pid, SIGKILL);
} else {
kill(worker_pid, SIGKILL); // Or something less violent if you prefer
}
wait(NULL); // Collect the other process
_exit(0); // Or some more informative status
}
waitpid(intermediate_pid, 0, 0);
Surprisingly simple :)
You can even leave out the intermediate child if you're sure no other module in the program is spwaning child processes of its own.
This is an interesting question.
I found sigtimedwait can do it.
EDIT 2016/08/29:
Thanks for Mark Edington's suggestion. I'v tested your example on Ubuntu 16.04, it works as expected.
Note: this only works for child processes. It's a pity that seems no equivalent way of Window's WaitForSingleObject(unrelated_process_handle, timeout) in Linux/Unix to get notified of unrelated process's termination within timeout.
OK, Mark Edington's sample code is here:
/* The program creates a child process and waits for it to finish. If a timeout
* elapses the child is killed. Waiting is done using sigtimedwait(). Race
* condition is avoided by blocking the SIGCHLD signal before fork().
*/
#include <sys/types.h>
#include <sys/wait.h>
#include <signal.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
static pid_t fork_child (void)
{
int p = fork ();
if (p == -1) {
perror ("fork");
exit (1);
}
if (p == 0) {
puts ("child: sleeping...");
sleep (10);
puts ("child: exiting");
exit (0);
}
return p;
}
int main (int argc, char *argv[])
{
sigset_t mask;
sigset_t orig_mask;
struct timespec timeout;
pid_t pid;
sigemptyset (&mask);
sigaddset (&mask, SIGCHLD);
if (sigprocmask(SIG_BLOCK, &mask, &orig_mask) < 0) {
perror ("sigprocmask");
return 1;
}
pid = fork_child ();
timeout.tv_sec = 5;
timeout.tv_nsec = 0;
do {
if (sigtimedwait(&mask, NULL, &timeout) < 0) {
if (errno == EINTR) {
/* Interrupted by a signal other than SIGCHLD. */
continue;
}
else if (errno == EAGAIN) {
printf ("Timeout, killing child\n");
kill (pid, SIGKILL);
}
else {
perror ("sigtimedwait");
return 1;
}
}
break;
} while (1);
if (waitpid(pid, NULL, 0) < 0) {
perror ("waitpid");
return 1;
}
return 0;
}
If your program runs only on contemporary Linux kernels (5.3 or later), the preferred way is to use pidfd_open (https://lwn.net/Articles/789023/ https://man7.org/linux/man-pages/man2/pidfd_open.2.html).
This system call returns a file descriptor representing a process, and then you can select, poll or epoll it, the same way you wait on other types of file descriptors.
For example,
int fd = pidfd_open(pid, 0);
struct pollfd pfd = {fd, POLLIN, 0};
poll(&pfd, 1, 1000) == 1;
The function can be interrupted with a signal, so you could set a timer before calling waitpid() and it will exit with an EINTR when the timer signal is raised. Edit: It should be as simple as calling alarm(5) before calling waitpid().
I thought that select will return EINTR when SIGCHLD signaled by on of the child.
I belive this should work:
while(1)
{
int retval = select(0, NULL, NULL, NULL, &tv, &mask);
if (retval == -1 && errno == EINTR) // some signal
{
pid_t pid = (waitpid(-1, &st, WNOHANG) == 0);
if (pid != 0) // some child signaled
}
else if (retval == 0)
{
// timeout
break;
}
else // error
}
Note: you can use pselect to override current sigmask and avoid interrupts from unneeded signals.
Instead of calling waitpid() directly, you could call sigtimedwait() with SIGCHLD (which would be sended to the parent process after child exited) and wait it be delived to the current thread, just as the function name suggested, a timeout parameter is supported.
please check the following code snippet for detail
static bool waitpid_with_timeout(pid_t pid, int timeout_ms, int* status) {
sigset_t child_mask, old_mask;
sigemptyset(&child_mask);
sigaddset(&child_mask, SIGCHLD);
if (sigprocmask(SIG_BLOCK, &child_mask, &old_mask) == -1) {
printf("*** sigprocmask failed: %s\n", strerror(errno));
return false;
}
timespec ts;
ts.tv_sec = MSEC_TO_SEC(timeout_ms);
ts.tv_nsec = (timeout_ms % 1000) * 1000000;
int ret = TEMP_FAILURE_RETRY(sigtimedwait(&child_mask, NULL, &ts));
int saved_errno = errno;
// Set the signals back the way they were.
if (sigprocmask(SIG_SETMASK, &old_mask, NULL) == -1) {
printf("*** sigprocmask failed: %s\n", strerror(errno));
if (ret == 0) {
return false;
}
}
if (ret == -1) {
errno = saved_errno;
if (errno == EAGAIN) {
errno = ETIMEDOUT;
} else {
printf("*** sigtimedwait failed: %s\n", strerror(errno));
}
return false;
}
pid_t child_pid = waitpid(pid, status, WNOHANG);
if (child_pid != pid) {
if (child_pid != -1) {
printf("*** Waiting for pid %d, got pid %d instead\n", pid, child_pid);
} else {
printf("*** waitpid failed: %s\n", strerror(errno));
}
return false;
}
return true;
}
Refer: https://android.googlesource.com/platform/frameworks/native/+/master/cmds/dumpstate/DumpstateUtil.cpp#46
If you're going to use signals anyways (as per Steve's suggestion), you can just send the signal manually when you want to exit. This will cause waitpid to return EINTR and the thread can then exit. No need for a periodic alarm/restart.
Due to circumstances I absolutely needed this to run in the main thread and it was not very simple to use the self-pipe trick or eventfd because my epoll loop was running in another thread. So I came up with this by scrounging together other stack overflow handlers. Note that in general it's much safer to do this in other ways but this is simple. If anyone cares to comment about how it's really really bad then I'm all ears.
NOTE: It is absolutely necessary to block signals handling in any thread save for the one you want to run this in. I do this by default as I believe it messy to handle signals in random threads.
static void ctlWaitPidTimeout(pid_t child, useconds_t usec, int *timedOut) {
int rc = -1;
static pthread_mutex_t alarmMutex = PTHREAD_MUTEX_INITIALIZER;
TRACE("ctlWaitPidTimeout: waiting on %lu\n", (unsigned long) child);
/**
* paranoid, in case this was called twice in a row by different
* threads, which could quickly turn very messy.
*/
pthread_mutex_lock(&alarmMutex);
/* set the alarm handler */
struct sigaction alarmSigaction;
struct sigaction oldSigaction;
sigemptyset(&alarmSigaction.sa_mask);
alarmSigaction.sa_flags = 0;
alarmSigaction.sa_handler = ctlAlarmSignalHandler;
sigaction(SIGALRM, &alarmSigaction, &oldSigaction);
/* set alarm, because no alarm is fired when the first argument is 0, 1 is used instead */
ualarm((usec == 0) ? 1 : usec, 0);
/* wait for the child we just killed */
rc = waitpid(child, NULL, 0);
/* if errno == EINTR, the alarm went off, set timedOut to true */
*timedOut = (rc == -1 && errno == EINTR);
/* in case we did not time out, unset the current alarm so it doesn't bother us later */
ualarm(0, 0);
/* restore old signal action */
sigaction(SIGALRM, &oldSigaction, NULL);
pthread_mutex_unlock(&alarmMutex);
TRACE("ctlWaitPidTimeout: timeout wait done, rc = %d, error = '%s'\n", rc, (rc == -1) ? strerror(errno) : "none");
}
static void ctlAlarmSignalHandler(int s) {
TRACE("ctlAlarmSignalHandler: alarm occured, %d\n", s);
}
EDIT: I've since transitioned to using a solution that integrates well with my existing epoll()-based eventloop, using timerfd. I don't really lose any platform-independence since I was using epoll anyway, and I gain extra sleep because I know the unholy combination of multi-threading and UNIX signals won't hurt my program again.
I can use a signal handler for SIGCHLD, and in the signal handler do whatever I was going to do when a child exits, or send a message to a different thread to do some action. But using a signal handler obfuscates the flow of the code a little bit.
In order to avoid race conditions you should avoid doing anything more complex than changing a volatile flag in a signal handler.
I think the best option in your case is to send a signal to the parent. waitpid() will then set errno to EINTR and return. At this point you check for waitpid return value and errno, notice you have been sent a signal and take appropriate action.
If a third party library is acceptable then the libkqueue project emulates kqueue (the *BSD eventing system) and provides basic process monitoring with EVFILT_PROC + NOTE_EXIT.
The main advantages of using kqueue or libkqueue is that it's cross platform, and doesn't have the complexity of signal handling. If your program is utilises async I/O you may also find it a lower friction interface than using something like epoll and the various *fd functions (signalfd, eventfd, pidfd etc...).
#include <stdio.h>
#include <stdint.h>
#include <sys/event.h> /* kqueue header */
#include <sys/types.h> /* for pid_t */
/* Link with -lkqueue */
int waitpid_timeout(pid_t pid, struct timespec *timeout)
{
struct kevent changelist, eventlist;
int kq, ret;
/* Populate a changelist entry (an event we want to be notified of) */
EV_SET(&changelist, pid, EVFILT_PROC, EV_ADD, NOTE_EXIT, 0, NULL);
kq = kqueue();
/* Call kevent with a timeout */
ret = kevent(kq, &changelist, 1, &eventlist, 1, timeout);
/* Kevent returns 0 on timeout, the number of events that occurred, or -1 on error */
switch (ret) {
case -1:
printf("Error %s\n", strerror(errno));
break;
case 0:
printf("Timeout\n");
break;
case 1:
printf("PID %u exited, status %u\n", (unsigned int)eventlist.ident, (unsigned int)eventlist.data);
break;
}
close(kq);
return ret;
}
Behind the scenes on Linux libkqueue uses either pidfd on Linux kernels >= 5.3 or a waiter thread that listens for SIGCHLD and notifies one or more kqueue instances when a process exits. The second approach is not efficient (it scans PIDs that interest has been registered for using waitid), but that doesn't matter unless you're waiting on large numbers of PIDs.
EVFILT_PROC support has been included in kqueue since its inception, and in libkqueue since v2.5.0.