I'm writing a program, and once a button is pushed, I have to execute a server process (that will stop only if I decide to kill him).
To execute this process, I decided to use fork/execv mechanism :
void Command::RunServer() {
pid = fork();
if (pid==0) {
chdir("./bin");
char str[10];
sprintf(str,"%d",port);
char *argv[] = {"./Server", str};
execv("./Server",argv);
}
else {
config->pid = pid;
return;
}
}
And in the method "button pushed", I do:
command->RunServer();
It seemed to work nicely a few days ago... and now i get error :
main: xcb_io.c:221: poll_for_event: Assertion `(((long) (event_sequence) - (long) (dpy->request)) <= 0)' failed.
Should I try to switch to pthread? Did I do something bad?
Thanks,
eo
When you do fork() all file descriptors of your process are duplicated in the new one. And when you do exec*() all file descriptors are also kept, unless they are marked with the flag FD_CLOEXEC.
My guess is that some fd used by some library (Xlib, probably) is inherited by the new process, and that the duplication is causing chaos in your program.
In these cases is useful the BSD function closefrom() (closefrom(3)) if you want to keep the standard I/O opened. Unfortunately, in linux there is no such function, so you have to do a close-all loop or similar cruft:
int open_max = sysconf (_SC_OPEN_MAX);
for (int i = 3; i < open_max; i++)
close(i);
You can read more about this problem here.
In the call to execv, argv has to be terminated by a null pointer.
The preceding line should be:
char* argv[] = { "./Server", str, NULL };
Related
I am trying to learn all interactions about signals and I discovered a funny interaction in it I can't understand.
Here's an abstract of the program, Im instructed to do execvp with grandchild, while child needs to wait for grandchild to finish. It runs correctly when without any signal interactions.
void say_Hi(int num) { printf("Finished\n"); }
int main() {
int i = 2;
char *command1[] = {"sleep", "5", NULL};
char *command2[] = {"sleep", "10", NULL};
signal(SIGCHLD, SIG_IGN);
signal(SIGUSR1, say_Hi);
while(i > 0) {
pid_t pid = fork();
if (pid == 0) {
pid_t pidChild = fork();
if (pidChild == 0) {
if (i == 2) {
execvp(command1[0], command1);
} else {
execvp(command2[0], command2);
}
} else if (pidChild > 0) {
waitpid(pidChild, 0, 0);
// kill(pid, SIGUSR1);
printf("pid finished: %d\n", pidChild);
exit(EXIT_FAILURE);
}
exit(EXIT_FAILURE);
} else {
//parent immediately goes to next loop
i--;
}
}
cin >> i; //just for me to pause and observate answers above
return 0;
}
As shown above, kill(pid, SIGUSR1); is commented, the program runs correctly.
Output：
pid finished: 638532 //after 5 sec
pid finished: 638533 //after 10 sec
However, when it is uncommented. Output becomes:
Finished
pid finished: 638610 //after 5 sec
Finished
Finished
Finished
Finished
pid finished: 638611 //after 5 sec too, why?
Finished
I would like to ask:
The whole program finished at once after 5 secs and a total of 6 "Finished" is printed out. Why is so?
Is there a way for me to modify it so that say_Hi function run in a total of two times only, in a correct time interval?
Please forgive me if my code looks stupid or bulky, I'm a newbie in programming. Any Comments about my code and help are appreciated!
void say_Hi(int num) { printf("Finished\n"); }
printf cannot be called in a signal handler. None of the C or the C++ library functions (with few exceptions) can be called in the signal handler. You can't even allocate or delete any memory from a signal handler (using either the C or the C++ library), except by using low-level OS calls like brk() or sbrk(). This is because of a very simple reason: that none of the C or the C++ library functions are signal-safe (with very few exceptions). Only function calls that are explicitly designated as "signal-safe" can be called from a signal handler. None of the C or C++ library functions or classes (with few exceptions) are signal-safe. The End.
The only thing that can be called from a signal handler are low-level operating system calls, like read() and write(), that operate directly on file handles. They are, by definition, signal-safe.
For this simple reason the shown code, when it comes to signals, is undefined behavior. Trying to analyze or figure out your programs behavior, from that respect, such as why or why not you see this message, is completely pointless. It cannot be logically analyzed. This is undefined behavior.
Answer:
kill(getpid(), SIG_USR1);
I am wondering what the best practice is for executing new processes (programs) from a running process. To be more specific, I am implementing a C/C++ job scheduler that has to run multiple binaries while communicating with them. Is exec or fork common? Or is there any library taking care of this?
You can use popen() to spawn the processes and communicate with them. In order to handle communication with many processes from a single parent process, use select() or poll() to multiplex the reading/writing of the file descriptors given to you by popen() (you can use fileno() to turn a FILE* into an integer file descriptor).
If you want a library to abstract much of this for you, I suggest libuv. Here's a complete example program I whipped up, largely following the docs at https://nikhilm.github.io/uvbook/processes.html#spawning-child-processes:
#include <cstdio>
#include <cstdlib>
#include <inttypes.h>
#include <uv.h>
static void alloc_buffer(uv_handle_t *handle, size_t suggested_size, uv_buf_t *buf)
{
*buf = uv_buf_init((char*)malloc(suggested_size), suggested_size);
}
void echo_read(uv_stream_t *server, ssize_t nread, const uv_buf_t* buf)
{
if (nread == -1) {
fprintf(stderr, "error echo_read");
return;
}
puts(buf->base);
}
static void on_exit(uv_process_t *req, int64_t exit_status, int term_signal)
{
fprintf(stderr, "Process %d exited with status %" PRId64 ", signal %d\n",
req->pid, exit_status, term_signal);
uv_close((uv_handle_t*)req, NULL);
}
int main()
{
uv_loop_t* loop = uv_default_loop();
const int N = 3;
uv_pipe_t channel[N];
uv_process_t child_req[N];
for (int ii = 0; ii < N; ++ii) {
char* args[3];
args[0] = const_cast<char*>("ls");
args[1] = const_cast<char*>(".");
args[2] = NULL;
uv_pipe_init(loop, &channel[ii], 1);
uv_stdio_container_t child_stdio[3]; // {stdin, stdout, stderr}
child_stdio[STDIN_FILENO].flags = UV_IGNORE;
child_stdio[STDOUT_FILENO].flags = uv_stdio_flags(UV_CREATE_PIPE | UV_WRITABLE_PIPE);
child_stdio[STDOUT_FILENO].data.stream = (uv_stream_t*)&channel[ii];
child_stdio[STDERR_FILENO].flags = UV_IGNORE;
uv_process_options_t options = {};
options.exit_cb = on_exit;
options.file = "ls";
options.args = args;
options.stdio = child_stdio;
options.stdio_count = sizeof(child_stdio) / sizeof(child_stdio[0]);
int r;
if ((r = uv_spawn(loop, &child_req[ii], &options))) {
fprintf(stderr, "%s\n", uv_strerror(r));
return EXIT_FAILURE;
} else {
fprintf(stderr, "Launched process with ID %d\n", child_req[ii].pid);
uv_read_start((uv_stream_t*)&channel[ii], alloc_buffer, echo_read);
}
}
return uv_run(loop, UV_RUN_DEFAULT);
}
The above will spawn three copies of ls to print the contents of the current directory. They all run asynchronously.
Okay let's start..
There are few ways to create another parallel task from one task. Although I wouldn't name all of them as processes.
Using fork() system call
Now as you have already mentioned that fork() creates a process from your parent process. There are few good things and few bad things about fork().
Good things
fork() is able to create a completely different process & in multi-core CPU systems, it can truly achieve parallelism
fork() also creates a child process with different pid & hence it is nice if you ever want to kill that process explicitly.
wait() & waitpid() system calls are nice to make the parent wait for child.
fork generates SIGCHILD signal and with sigaction function you can make the parent wait for child without blocking it.
Bad things
fork processes do not share the same address space & hence if one process is having say a variable var, the other process cannot access directly that same var. Hence communication is a big issue.
To communicate you need to use certain IPC mechanisms like pipe, namedpipe, messageQueues or sharedMemory
Now out of these pipe, namedpipe and messageQueues can use read & write system calls and because read & write system calls are blocking system calls, you application remains synchronized but these IPCs are very slow. The only fast IPC is sharedMemory but it cannot use read & write & hence you need to use your own synchronization mechanisms, like semaphores. But implementing semaphores for bigger applications is difficult.
Here comes pthread
Now thread removes all the difficulties that are faced by fork.
It doesn't create a separate process.
It rather creates few light-weight subtasks which can run almost parallel.
They all share same address space & hence no need for any IPC.
The come with mutex which is wonderful for any synchronizations needed even for bigger applications.
Thread also don't create any process hence all threads is a part of same process and hence will have same pid.
Note: In C++, thread is a part of C++ library, not a system call.
Note 2: Boost threads in C++ are much more mature & recommended to use.
The main idea although is to know that when to use thread & when to use process.
If you need to create a sub-task which doesn't need to work with some other task but it has to work in isolation, use process; otherwise use thread.
The exec family syscalls are different. It uses your same pid. Hence if you create an application with 500 lines say, and you get a exec call at line number 250, then that exec process will be pasted on your whole process and after exec call, you program will not resume from 251 line. Also, exec calls don't flush your stdio buffers.
But yes, if you intend to create a separate process, and then use exec call to perform that task and then come out, then you are welcome to do it, but remember the IPC to store the output otherwise it is of no use
For more info on fork click here
For more info on thread click here
For boost therad click here
#John Zwinck answer is also good but I know little about select() system call but yes it is possible that way too
Edited: As # Jonathan Leffler pointed
Editing after a long: After some years I now never think of using all these SPOOKY libraries or senseless gruesome ways of parallel or should I say SEEMINGLY parallel processing. Enter coroutines, the future of CONCURRENT processing. Look at the following Go code. Sure this is possible in C/C++ too. This code would hardly be few milliseconds slower for 7.7 mil rows in database than its C/C++ thread based implementation but sever times more manageable and scalable.
package main
import (
"fmt"
"reflect"
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/sqlite"
)
type AirQuality struct {
// gorm.Model
// ID uint `gorm:"column:id"`
Index string `gorm:"column:index"`
BEN string `gorm:"column:BEN"`
CH4 string `gorm:"column:CH4"`
CO string `gorm:"column:CO"`
EBE string `gorm:"column:EBE"`
MXY string `gorm:"column:MXY"`
NMHC string `gorm:"column:NMHC"`
NO string `gorm:"column:NO"`
NO2 string `gorm:"column:NO_2"`
NOX string `gorm:"column:NOx"`
OXY string `gorm:"column:OXY"`
O3 string `gorm:"column:O_3"`
PM10 string `gorm:"column:PM10"`
PM25 string `gorm:"column:PM25"`
PXY string `gorm:"column:PXY"`
SO2 string `gorm:"column:SO_2"`
TCH string `gorm:"column:TCH"`
TOL string `gorm:"column:TOL"`
Time string `gorm:"column:date; type:timestamp"`
Station string `gorm:"column:station"`
}
func (AirQuality) TableName() string {
return "AQ"
}
func main() {
c := generateRowsConcurrent("boring!!")
for row := range c {
fmt.Println(row)
}
}
func generateRowsConcurrent(msg string) <-chan []string {
c := make(chan []string)
go func() {
db, err := gorm.Open("sqlite3", "./load_testing_7.6m.db")
if err != nil {
panic("failed to connect database")
}
defer db.Close()
rows, err := db.Model(&AirQuality{}).Limit(20).Rows()
defer rows.Close()
if err != nil {
panic(err)
}
for rows.Next() {
var aq AirQuality
db.ScanRows(rows, &aq)
v := reflect.Indirect(reflect.ValueOf(aq))
var buf []string
for i := 0; i < v.NumField(); i++ {
buf = append(buf, v.Field(i).String())
}
c <- buf
}
defer close(c)
}()
return c
}
There is mostly nothing found through search about calling main - so I am going to guess this question is that fundamentally 'off'.
The suggested questions provides this C# question "Calling Main() from another class" the answer there is that you don't, use a subfunction under Main() and call that; so my assumption is that the same answer applies here with fork().
void somefunction ()
{
pid_t pid;
pid = fork();
if (pid == 0) {
char *p;
char *argv[] = { (char*)spawn_count, (char*)gradiant, (char*)i, (char*)(i+spread), p };
main(5, **argv);
}
else if (pid == -1)
cout << "ERROR: can't fork" << endl;
else ; //PID == 1,
}
This compiles in g++ with "error: 'main' was not declared in this scope"
Questions:
How is main(argc, argv) scoped? I can follow the process in windows with LPTSTR.
After fork(), to start the child from main(), how is this done?
After fork(), where is the child, in the same function that called child, in main?
Why do you want to call main?
Could you not just fork in main something like below:
int main()
{
pid_t pid;
pid = fork();
if(pid < 0 )
{
// Error some issue forking
}
if (pid == 0) {
// Child related processing here
}
else {
// parent related processing here
}
}
Fork returns twice once for parent and once for child and at the same point from where the fork system call was made.
You could refer this link to read more: http://www.yolinux.com/TUTORIALS/ForkExecProcesses.html
For anyone who falls into the same trap I did - Linux and Windows parallel processing works very differently.
In Windows, you call the program again and pass arguments through main()
In Linux, you call fork() then call any function you want from within your program, just like with any other function. This approach is super easy compared to the 'crap' you go through to fork windows.
After understanding this distinction, I immediately removed windows from my laptop and installed Linux :)
I have the helper function below, used to execute a command and get the return value on posix systems. I used to use popen, but it is impossible to get the return code of an application with popen if it runs and exits before popen/pclose gets a chance to do its work.
The following helper function creates a process fork, uses execvp to run the desired external process, and then the parent uses waitpid to get the return code. I'm seeing odd cases where it's refusing to run.
When called with wait = true, waitpid should return the exit code of the application no matter what. However, I'm seeing stdout output that specifies the return code should be non-zero, yet the return code is zero. Testing the external process in a regular shell, then echoing $? returns non-zero, so it's not a problem w/ the external process not returning the right code. If it's of any help, the external process being run is mount(8) (yes, I know I can use mount(2) but that's besides the point).
I apologize in advance for a code dump. Most of it is debugging/logging:
inline int ForkAndRun(const std::string &command, const std::vector<std::string> &args, bool wait = false, std::string *output = NULL)
{
std::string debug;
std::vector<char*> argv;
for(size_t i = 0; i < args.size(); ++i)
{
argv.push_back(const_cast<char*>(args[i].c_str()));
debug += "\"";
debug += args[i];
debug += "\" ";
}
argv.push_back((char*)NULL);
neosmart::logger.Debug("Executing %s", debug.c_str());
int pipefd[2];
if (pipe(pipefd) != 0)
{
neosmart::logger.Error("Failed to create pipe descriptor when trying to launch %s", debug.c_str());
return EXIT_FAILURE;
}
pid_t pid = fork();
if (pid == 0)
{
close(pipefd[STDIN_FILENO]); //child isn't going to be reading
dup2(pipefd[STDOUT_FILENO], STDOUT_FILENO);
close(pipefd[STDOUT_FILENO]); //now that it's been dup2'd
dup2(pipefd[STDOUT_FILENO], STDERR_FILENO);
if (execvp(command.c_str(), &argv[0]) != 0)
{
exit(EXIT_FAILURE);
}
return 0;
}
else if (pid < 0)
{
neosmart::logger.Error("Failed to fork when trying to launch %s", debug.c_str());
return EXIT_FAILURE;
}
else
{
close(pipefd[STDOUT_FILENO]);
int exitCode = 0;
if (wait)
{
waitpid(pid, &exitCode, wait ? __WALL : (WNOHANG | WUNTRACED));
std::string result;
char buffer[128];
ssize_t bytesRead;
while ((bytesRead = read(pipefd[STDIN_FILENO], buffer, sizeof(buffer)-1)) != 0)
{
buffer[bytesRead] = '\0';
result += buffer;
}
if (wait)
{
if ((WIFEXITED(exitCode)) == 0)
{
neosmart::logger.Error("Failed to run command %s", debug.c_str());
neosmart::logger.Info("Output:\n%s", result.c_str());
}
else
{
neosmart::logger.Debug("Output:\n%s", result.c_str());
exitCode = WEXITSTATUS(exitCode);
if (exitCode != 0)
{
neosmart::logger.Info("Return code %d", (exitCode));
}
}
}
if (output)
{
result.swap(*output);
}
}
close(pipefd[STDIN_FILENO]);
return exitCode;
}
}
Note that the command is run OK with the correct parameters, the function proceeds without any problems, and WIFEXITED returns TRUE. However, WEXITSTATUS returns 0, when it should be returning something else.
Probably isn't your main issue, but I think I see a small problem. In your child process, you have...
dup2(pipefd[STDOUT_FILENO], STDOUT_FILENO);
close(pipefd[STDOUT_FILENO]); //now that it's been dup2'd
dup2(pipefd[STDOUT_FILENO], STDERR_FILENO); //but wait, this pipe is closed!
But I think what you want is:
dup2(pipefd[STDOUT_FILENO], STDOUT_FILENO);
dup2(pipefd[STDOUT_FILENO], STDERR_FILENO);
close(pipefd[STDOUT_FILENO]); //now that it's been dup2'd for both, can close
I don't have much experience with forks and pipes in Linux, but I did write a similar function pretty recently. You can take a look at the code to compare, if you'd like. I know that my function works.
execAndRedirect.cpp
I'm using the mongoose library, and grepping my code for SIGCHLD revealed that using mg_start from mongoose results in setting SIGCHLD to SIG_IGN.
From the waitpid man page, on Linux a SIGCHLD set to SIG_IGN will not create a zombie process, so waitpid will fail if the process has already successfully run and exited - but will run OK if it hasn't yet. This was the cause of the sporadic failure of my code.
Simply re-setting SIGCHLD after calling mg_start to a void function that does absolutely nothing was enough to keep the zombie records from being immediately erased.
Per #Geoff_Montee's advice, there was a bug in my redirect of STDERR, but this was not responsible for the problem as execvp does not store the return value in STDERR or even STDOUT, but rather in the kernel object associated with the parent process (the zombie record).
#jilles' warning about non-contiguity of vector in C++ does not apply for C++03 and up (only valid for C++98, though in practice, most C++98 compilers did use contiguous storage, anyway) and was not related to this issue. However, the advice on reading from the pipe before blocking and checking the output of waitpid is spot-on.
I've found that pclose does NOT block and wait for the process to end, contrary to the documentation (this is on CentOS 6). I've found that I need to call pclose and then call waitpid(pid,&status,0); to get the true return value.
This question already has answers here:
How do I execute a command and get the output of the command within C++ using POSIX?
(12 answers)
Closed 7 years ago.
I'm trying to start an external application through system() - for example, system("ls"). I would like to capture its output as it happens so I can send it to another function for further processing. What's the best way to do that in C/C++?
From the popen manual:
#include <stdio.h>
FILE *popen(const char *command, const char *type);
int pclose(FILE *stream);
Try the popen() function. It executes a command, like system(), but directs the output into a new file. A pointer to the stream is returned.
FILE *lsofFile_p = popen("lsof", "r");
if (!lsofFile_p)
{
return -1;
}
char buffer[1024];
char *line_p = fgets(buffer, sizeof(buffer), lsofFile_p);
pclose(lsofFile_p);
EDIT: misread question as wanting to pass output to another program, not another function. popen() is almost certainly what you want.
System gives you full access to the shell. If you want to continue using it, you can
redirect it's output to a temporary file, by system("ls > tempfile.txt"), but choosing a secure temporary file is a pain. Or, you can even redirect it through another program: system("ls | otherprogram");
Some may recommend the popen() command. This is what you want if you can process the output yourself:
FILE *output = popen("ls", "r");
which will give you a FILE pointer you can read from with the command's output on it.
You can also use the pipe() call to create a connection in combination with fork() to create new processes, dup2() to change the standard input and output of them, exec() to run the new programs, and wait() in the main program to wait for them. This is just setting up the pipeline much like the shell would. See the pipe() man page for details and an example.
The functions popen() and such don't redirect stderr and such; I wrote popen3() for that purpose.
Here's a bowdlerised version of my popen3():
int popen3(int fd[3],const char **const cmd) {
int i, e;
int p[3][2];
pid_t pid;
// set all the FDs to invalid
for(i=0; i<3; i++)
p[i][0] = p[i][1] = -1;
// create the pipes
for(int i=0; i<3; i++)
if(pipe(p[i]))
goto error;
// and fork
pid = fork();
if(-1 == pid)
goto error;
// in the parent?
if(pid) {
// parent
fd[STDIN_FILENO] = p[STDIN_FILENO][1];
close(p[STDIN_FILENO][0]);
fd[STDOUT_FILENO] = p[STDOUT_FILENO][0];
close(p[STDOUT_FILENO][1]);
fd[STDERR_FILENO] = p[STDERR_FILENO][0];
close(p[STDERR_FILENO][1]);
// success
return 0;
} else {
// child
dup2(p[STDIN_FILENO][0],STDIN_FILENO);
close(p[STDIN_FILENO][1]);
dup2(p[STDOUT_FILENO][1],STDOUT_FILENO);
close(p[STDOUT_FILENO][0]);
dup2(p[STDERR_FILENO][1],STDERR_FILENO);
close(p[STDERR_FILENO][0]);
// here we try and run it
execv(*cmd,const_cast<char*const*>(cmd));
// if we are there, then we failed to launch our program
perror("Could not launch");
fprintf(stderr," \"%s\"\n",*cmd);
_exit(EXIT_FAILURE);
}
// preserve original error
e = errno;
for(i=0; i<3; i++) {
close(p[i][0]);
close(p[i][1]);
}
errno = e;
return -1;
}
The most efficient way is to use stdout file descriptor directly, bypassing FILE stream:
pid_t popen2(const char *command, int * infp, int * outfp)
{
int p_stdin[2], p_stdout[2];
pid_t pid;
if (pipe(p_stdin) == -1)
return -1;
if (pipe(p_stdout) == -1) {
close(p_stdin[0]);
close(p_stdin[1]);
return -1;
}
pid = fork();
if (pid < 0) {
close(p_stdin[0]);
close(p_stdin[1]);
close(p_stdout[0]);
close(p_stdout[1]);
return pid;
} else if (pid == 0) {
close(p_stdin[1]);
dup2(p_stdin[0], 0);
close(p_stdout[0]);
dup2(p_stdout[1], 1);
dup2(::open("/dev/null", O_WRONLY), 2);
/// Close all other descriptors for the safety sake.
for (int i = 3; i < 4096; ++i) {
::close(i);
}
setsid();
execl("/bin/sh", "sh", "-c", command, NULL);
_exit(1);
}
close(p_stdin[0]);
close(p_stdout[1]);
if (infp == NULL) {
close(p_stdin[1]);
} else {
*infp = p_stdin[1];
}
if (outfp == NULL) {
close(p_stdout[0]);
} else {
*outfp = p_stdout[0];
}
return pid;
}
To read output from child use popen2() like this:
int child_stdout = -1;
pid_t child_pid = popen2("ls", 0, &child_stdout);
if (!child_pid) {
handle_error();
}
char buff[128];
ssize_t bytes_read = read(child_stdout, buff, sizeof(buff));
To both write and read:
int child_stdin = -1;
int child_stdout = -1;
pid_t child_pid = popen2("grep 123", &child_stdin, &child_stdout);
if (!child_pid) {
handle_error();
}
const char text = "1\n2\n123\n3";
ssize_t bytes_written = write(child_stdin, text, sizeof(text) - 1);
char buff[128];
ssize_t bytes_read = read(child_stdout, buff, sizeof(buff));
The functions popen() and pclose() could be what you're looking for.
Take a look at the glibc manual for an example.
In Windows, instead of using system(), use CreateProcess, redirect the output to a pipe and connect to the pipe.
I'm guessing this is also possible in some POSIX way?
Actually, I just checked, and:
popen is problematic, because the process is forked. So if you need to wait for the shell command to execute, then you're in danger of missing it. In my case, my program closed even before the pipe got to do it's work.
I ended up using system call with tar command on linux. The return value from system was the result of tar.
So: if you need the return value, then not no only is there no need to use popen, it probably won't do what you want.
In this page: capture_the_output_of_a_child_process_in_c describes the limitations of using popen vs. using fork/exec/dup2/STDOUT_FILENO approach.
I'm having problems capturing tshark output with popen.
And I'm guessing that this limitation might be my problem:
It returns a stdio stream as opposed to a raw file descriptor, which
is unsuitable for handling the output asynchronously.
I'll come back to this answer if I have a solution with the other approach.
I'm not entirely certain that its possible in standard C, as two different processes don't typically share memory space. The simplest way I can think of to do it would be to have the second program redirect its output to a text file (programname > textfile.txt) and then read that text file back in for processing. However, that may not be the best way.