I have a multiprocessing application that works well, except the parent process seems to exit twice.
I left out some of the code for simplification. Basically, I use libcurl (I wrote my own abstraction layer for it) to get JSON data from a server (left the code for this out) and then the simdjson library to iterate through it and run worker processes where required.
At the end I wait for all child processes (in the parent process) to terminate before printing "done". I can see however, that my program is printing "done" twice. I presume once after it's done in the for loop to create all the worker processes and then again once the last child returns. At least that is what I can see from the output on the console, as the child processes print to the console as well. However, given that I use if (pid_fork > 0), i.e. I must be in the parent process, any subsequent code should be executed only once. What am I doing wrong?
#include <iostream>
#include <vector>
#include <unistd.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
#include "simdjson.h"
int main (int argc, char *argv[])
{
/* some other code */
pid_t pid_fork;
std::vector<int> v_pid;
// loop through json
for (simdjson::dom::element mq_item : json_mq_items)
{
pid_fork = fork();
if (pid_fork == -1)
{
std::cout << "error: could not fork process" << std::endl;
return EXIT_FAILURE;
} else if (pid_fork > 1) // parent process
{
v_pid.push_back(pid_fork);
}
else // child process (pid_fork == 0)
{
char *argv[] = { (char*)(std::string("foo")), NULL };
if (execv((static_cast<std::string>("./foo")).c_str(), argv) == -1)
{
std::cout << "could not load child" << std::endl;
return EXIT_FAILURE;
}
}
}
// in parent process only
if (pid_fork > 0)
{
// Wait for all child processes to terminate
for (size_t i = 0; i < v_pid.size(); i++)
{
while (waitpid(v_pid[i], NULL, 0) > 0);
}
/* some other code */
std::cout << "done" << std::endl;
return EXIT_SUCCESS;
}
}
Related
Following this documentation, I am testing how to stop and resume a process. I have basic code to test as follows:
#include <iostream>
#include <csignal>
#include <unistd.h>
int main() {
std::cout << "Hello" << std::endl;
int pid = getpid();
kill(pid, SIGSTOP);
kill(pid, SIGCONT);
std::cout << "Bye" << std::endl;
return 0;
}
The output is:
Hello
It stops the process, but it never resumes it. How should I fix it?
A solution, if a bit complicated, is to create a child process to start and stop the parent. Here is a small code example, that might help:
#include <iostream>
#include <csignal>
#include <unistd.h>
int pid; //Include declaration outside so it transfers to the child process
int main() {
std::cout << "Hello" << std::endl;
pid = getpid();
int returned_pid = fork(); //Duplicate process into 2 identical processes
if(returned_pid) {
// If it is the parent process, then fork returns the child process pid
// This is executed by the parent process
usleep(1000); // Sleep a millisecond to allow for the stop command to run
} else {
// If fork returns 0, then it is the child process
// The else is executed by the child process
kill(pid, SIGSTOP); // Stop parent process
usleep(3000000); // Delay 3 seconds
kill(pid, SIGCONT); // Resume parent process
}
if(returned_pid) { // Only print if parent process
std::cout << "Bye" << std::endl;
}
return 0;
}
Clarification: The fork command returns 2 different values in the 2 processes: 0 in the child, and the pid of the child process in the parent.
Other note: When running this in a terminal, it will look weird, as the terminal may note that the process was stopped and give a new command line, but then the process resumes, so prints Bye over it. Just a note.
I am intending to set up a pipeline between two processes: parent and child. The parent forks the child and uses execve to replace its image with that of a specified process.
The parent reads from stdin via std::getline(std::cin, input_line).
The child writes to the stdout via std::cout << output_line.
I am looking to setup a pipe and redirect the output of the child to the input of the parent.
The problem is that the parent receives each input (where each input is a number output by the child on stdout) twice. I would like to fix this issue but I don't understand why it is happening.
Code is compiled with g++ 7.4.0 and C++11 standard version.
Child is compiled to a binary called 'p1'.
Parent code:
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
#include <iostream>
char *
const p1argv[] = {
(char * )
"./p1",
nullptr
};
char *
const p1envp[] = {
(char * ) nullptr
};
int main(int argc, char ** argv) {
pid_t p1id;
int p1fd[2];
pipe(p1fd);
if (p1id = fork() == 0) {
close(p1fd[0]);
dup2(p1fd[1], STDOUT_FILENO);
execve(argv[0], p1argv, p1envp);
perror("Error: failed to execve ./p1.");
} else {
dup2(p1fd[0], STDIN_FILENO);
close(p1fd[1]);
std::string line;
while (std::getline(std::cin, line)) {
std::cout << "d(" << line << ")" << std::endl;
}
int status;
waitpid(p1id, & status, 0);
close(p1fd[0]);
}
}
Child code:
#include <iostream>
#include <thread>
int main(int argc, char** argv) {
long it = 0;
while(true) {
it += 1;
std::cout << std::to_string(it) << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(2));
}
}
The actual output for the sample code is:
d(d(1))
d(d(2))
...
The expected output is:
d(1)
d(2)
...
The problem is that this line:
execve(argv[0], p1argv, p1envp);
Is re-executing the main parent program, because that is what the content of argv[0] is at this point. I think you want to find some way to specify "./p1" there.
I am learning fork and exec and creating multiple child processes using fork and execlp and all I do in the child process is let it sleep. Basically I just want all my child to be alive. But as soon as i start my monitor.cpp which creates processes all of the child exit immediately and they do defunct!
Monitor which forks multiple children
#include <iostream>
#include <thread>
#include <chrono>
#include <string>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
int main(int argc, char* argv[])
{
for(size_t i=0; i<std::stoi(argv[1]) ; ++i)
{
int pid = fork();
if(pid == 0)
{
execlp("child", "child", std::string(std::to_string(i)).c_str(), (char *)0);
std::cout << "child exiting " << std::endl;
exit(1);
}
else if(pid > 0)
{
std::cout <<"child started with " << pid << std::endl;
}
else
{
std::cout << "fork failed" << std::endl;
}
}
while(true)
{
std::this_thread::sleep_for(std::chrono::seconds(100000));
}
return 0;
}
Child Code
#include <iostream>
#include <thread>
#include <chrono>
int main(int argc, char* argv[])
{
std::cout << " child started with id " << argv[1] << std::endl;
std::cout <<"child sleeping " << argv[1] << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(1000));
std::cout << "child exiting " << argv[1] << std::endl;
return 0;
}
Output:
child started with 1834
child started with 1835
child exiting
child started with 1836
child exiting
child started with 1837
child started with 1838
child started with 1839
child exiting
child started with 1840
child started with 1841
child exiting
child started with 1842
child started with 1843
child exiting
child exiting
child exiting
child exiting
child exiting
child exiting
ps -ef shows all of my child processes as Defunct even though my parent is still alive.
Can you please explain what am I missing?
From the 'execlp' man page:
The exec() functions only return if an error has occurred. The return value is -1, and errno is set to indicate the error.
Since "child exiting" is being printed in two places, it's not obvious if it's exiting. You need to check it's return value and errno.
You need to reap the child-process as they exit. This is done using wait or waitpid calls.
Until the parent has done this, they will be visible as defunc / zombie processes. (init, process 1, is responsible for reaping all process that do not have a parent after they exit)
I'm trying to make a wrapper around an interactive program. For this I use the combination of pipe, dup2 and poll. All seems to go well until the child terminates. At this step the parent process appears to lose its stdin, which is what I can't seem to understand why.
Here's the code:
#include <unistd.h>
#include <sys/types.h>
#include <stdlib.h>
#include <stdio.h>
#include <poll.h>
#include <fcntl.h>
#include <signal.h>
#include <vector>
#include <string>
#include <iostream>
struct SystemFunctionFailure
{
std::string what;
SystemFunctionFailure(std::string const& what) : what(what) {}
};
template<typename T,size_t N> constexpr size_t countof(const T(&)[N]) { return N; }
void readAndPrint(std::string const& what, int fd)
{
std::cerr << "Reading "+what+"\n";
std::vector<char> buffer(1024);
const auto bytesRead=read(fd,buffer.data(),buffer.size());
if(bytesRead==-1)
{
if(errno!=EAGAIN)
throw SystemFunctionFailure("read "+what);
}
else if(bytesRead==0)
{
std::cerr << "EOF reached on "+what+"\n";
exit(0);
}
else
std::cerr << "CONTENTS OF "+what+": "+std::string(buffer.data(),buffer.size())+"\n";
}
int main()
{
try
{
int pipeChildOut[2];
if(pipe(pipeChildOut)==-1) throw SystemFunctionFailure("pipe for child stdout");
int pipeChildErr[2];
if(pipe(pipeChildErr)==-1) throw SystemFunctionFailure("pipe for child stderr");
int pipeChildIn[2];
if(pipe(pipeChildIn)==-1) throw SystemFunctionFailure("pipe for child stdin");
const auto child=fork();
if(child==-1) throw SystemFunctionFailure("fork");
if(child)
{
dup2(pipeChildOut[1],STDOUT_FILENO);
close(pipeChildOut[0]);
dup2(pipeChildErr[1],STDERR_FILENO);
close(pipeChildErr[0]);
dup2(pipeChildIn[0],STDIN_FILENO);
close(pipeChildIn[1]);
execlp("sh","sh","-c","sleep 1; echo Test ; sleep 1; echo Child is exiting... >&2",nullptr);
throw SystemFunctionFailure("execlp returned");
}
else
{
const int childStdErr=pipeChildErr[0];
const int childStdOut=pipeChildOut[0];
dup2(pipeChildIn[1],STDOUT_FILENO);
fcntl(childStdErr,F_SETFL,O_NONBLOCK);
fcntl(childStdOut,F_SETFL,O_NONBLOCK);
fcntl(STDIN_FILENO,F_SETFL,O_NONBLOCK);
while(true)
{
std::cerr << "New iteration of IO loop\n";
pollfd pollfds[]={ // making the indices coincide with .._FILENO
{STDIN_FILENO,POLLIN},
{childStdOut,POLLIN},
{childStdErr,POLLIN},
};
if(poll(pollfds,countof(pollfds),{-1})==-1)
throw SystemFunctionFailure("poll");
std::cerr << "poll returned\n";
for(unsigned i=0;i<countof(pollfds);++i)
std::cerr <<" pollfds["<<i<<"].revents: " << pollfds[i].revents << "\n";
if(pollfds[ STDIN_FILENO].revents&POLLIN) readAndPrint("stdin" ,pollfds[ STDIN_FILENO].fd);
if(pollfds[STDOUT_FILENO].revents&POLLIN) readAndPrint("stdout",pollfds[STDOUT_FILENO].fd);
if(pollfds[STDERR_FILENO].revents&POLLIN) readAndPrint("stderr",pollfds[STDERR_FILENO].fd);
}
}
}
catch(SystemFunctionFailure& ex)
{
perror(ex.what.c_str());
exit(EXIT_FAILURE);
}
}
Here the child closes its original stdin implicitly via dup2, so it seems it shouldn't affect any accesses of the parent to the console input. But for some reason here's what I get as output:
$ g++ test.cpp -o test -std=c++14 && ./test
New iteration of IO loop
poll returned
pollfds[0].revents: 0
pollfds[1].revents: 1
pollfds[2].revents: 0
Reading stdout
CONTENTS OF stdout: Test
New iteration of IO loop
poll returned
pollfds[0].revents: 0
pollfds[1].revents: 0
pollfds[2].revents: 1
Reading stderr
CONTENTS OF stderr: Child is exiting...
New iteration of IO loop
$
I.e. I get the shell prompt, so the parent isn't in the foreground anymore. After this, if I wait several seconds and type a letter, I get this output:
poll returned
pollfds[0].revents: 1
pollfds[1].revents: 0
pollfds[2].revents: 0
Reading stdin
read stdin: Input/output error
I'd like to at least have the parent process retain access to its console input after the child dies. After reading an answer to another question, I think my problem is related, but that answer doesn't answer my question: "How to do it right?".
I think if(child) should be if(child == 0).
From https://linux.die.net/man/2/fork
On success, the PID of the child process is returned in the parent,
and 0 is returned in the child. On failure, -1 is returned in the
parent, no child process is created, and errno is set appropriately.
I am forking a number of processes and I want to measure how long it takes to complete the whole task, that is when all processes forked are completed. Please advise how to make the parent process wait until all child processes are terminated? I want to make sure that I stop the timer at the right moment.
Here is as a code I use:
#include <iostream>
#include <string>
#include <fstream>
#include <sys/time.h>
#include <sys/wait.h>
using namespace std;
struct timeval first, second, lapsed;
struct timezone tzp;
int main(int argc, char* argv[])// query, file, num. of processes.
{
int pCount = 5; // process count
gettimeofday (&first, &tzp); //start time
pid_t* pID = new pid_t[pCount];
for(int indexOfProcess=0; indexOfProcess<pCount; indexOfProcess++)
{
pID[indexOfProcess]= fork();
if (pID[indexOfProcess] == 0) // child
{
// code only executed by child process
// magic here
// The End
exit(0);
}
else if (pID[indexOfProcess] < 0) // failed to fork
{
cerr << "Failed to fork" << endl;
exit(1);
}
else // parent
{
// if(indexOfProcess==pCount-1) and a loop with waitpid??
gettimeofday (&second, &tzp); //stop time
if (first.tv_usec > second.tv_usec)
{
second.tv_usec += 1000000;
second.tv_sec--;
}
lapsed.tv_usec = second.tv_usec - first.tv_usec;
lapsed.tv_sec = second.tv_sec - first.tv_sec;
cout << "Job performed in " <<lapsed.tv_sec << " sec and " << lapsed.tv_usec << " usec"<< endl << endl;
}
}//for
}//main
I'd move everything after the line "else //parent" down, outside the for loop. After the loop of forks, do another for loop with waitpid, then stop the clock and do the rest:
for (int i = 0; i < pidCount; ++i) {
int status;
while (-1 == waitpid(pids[i], &status, 0));
if (!WIFEXITED(status) || WEXITSTATUS(status) != 0) {
cerr << "Process " << i << " (pid " << pids[i] << ") failed" << endl;
exit(1);
}
}
gettimeofday (&second, &tzp); //stop time
I've assumed that if the child process fails to exit normally with a status of 0, then it didn't complete its work, and therefore the test has failed to produce valid timing data. Obviously if the child processes are supposed to be killed by signals, or exit non-0 return statuses, then you'll have to change the error check accordingly.
An alternative using wait:
while (true) {
int status;
pid_t done = wait(&status);
if (done == -1) {
if (errno == ECHILD) break; // no more child processes
} else {
if (!WIFEXITED(status) || WEXITSTATUS(status) != 0) {
cerr << "pid " << done << " failed" << endl;
exit(1);
}
}
}
This one doesn't tell you which process in sequence failed, but if you care then you can add code to look it up in the pids array and get back the index.
The simplest method is to do
while(wait() > 0) { /* no-op */ ; }
This will not work if wait() fails for some reason other than the fact that there are no children left. So with some error checking, this becomes
int status;
[...]
do {
status = wait();
if(status == -1 && errno != ECHILD) {
perror("Error during wait()");
abort();
}
} while (status > 0);
See also the manual page wait(2).
Call wait (or waitpid) in a loop until all children are accounted for.
In this case, all processes are synchronizing anyway, but in general wait is preferred when more work can be done (eg worker process pool), since it will return when the first available process state changes.
I believe the wait system call will accomplish what you are looking for.
for (int i = 0; i < pidCount; i++) {
while (waitpid(pids[i], NULL, 0) > 0);
}
It won't wait in the right order, but it will stop shortly after the last child dies.