I am trying to create a C++ program that computes some values in a simulation and then passes the output to a Python script for drawing. Both processes operate in an infinite loop, the C++ program computes steps in the simulation and passes it to the Python program via a pipe, the Python program repeatedly reads from the pipe and displays the data.
So far I've tried achieving this by forking the current process and executing the Python script under the child process however when I start the infinite loop in the parent process it blocks all IO and nothing is ever outputted to the screen until I terminate the parent process and turn the child process into a zombie. The C++ program that starts the script is as follows:
int main(int argc, char* argv[]) {
int pipe_to_py[2];
if (::pipe(pipe_to_py)) {
std::cout << "Failed to open pipe for uni-directional communication";
}
pid_t pid = fork();
// if pid is the child process then start up the python script and pass the file descriptors over to it
if (pid == 0) {
::close(pipe_to_py[1]); // close the "write end"
std::ostringstream oss;
oss << "export PY_READ_FD=" << std::to_string(pipe_to_py[0]) << " && "
<< "export PYTHONUNBUFFERED=true && "
<< "python3 script.py";
::system(oss.str().c_str()); // call py script
} else if (pid < 0) { exit(1);
} else {
::close(pipe_to_py[0]);
program_loop(&pipe_to_py[1]); // function with while(true) loop
}
return 0;
}
While the Python script is:
# the physical animation function draws data from the pipe and plots it
def animate(i):
data = read_data_from_pipe()
draw(data)
anim = FuncAnimation(fig, animate, interval=100)
fig.show()
plt.show()
I'm just wondering what exactly I'm doing wrong and why I/O is being blocked for the child process.
Related
A child process runs a bin file, which is provided by Qualcomm.
The child process is invoked by my parent process, which is developed by me.
When the child process is running, it always prints lots pieces of logs in shell command.
So, am I able to redirect Qualcomm's outstream from stdout to another file in the parent process?
As you know, it's nearly impossible to push Qualcomm to update this bin file.
The key piece here is the POSIX function dup2, which lets you essentially replace one file descriptor with another. And if you use fork (not system), you actually have control of what happens in the child process between the fork and the exec* that loads the other executable.
#include <cstdlib>
extern "C" {
#include <fcntl.h>
#include <unistd.h>
}
#include <stdexcept>
#include <iostream>
pid_t start_child(const char* program, const char* output_filename)
{
pid_t pid = fork();
if (pid < 0) {
// fork failed!
std::perror("fork");
throw std::runtime_error("fork failed");
} else if (pid == 0) {
// This code runs in the child process.
int output_fd = open(output_filename, O_WRONLY | O_CREAT | O_TRUNC);
if (output_fd < 0) {
std::cerr << "Failed to open log file " << output_filename << ":"
<< std::endl;
std::perror("open");
std::exit(1);
}
// Replace the child's stdout and stderr handles with the log file handle:
if (dup2(output_fd, STDOUT_FILENO) < 0) {
std::perror("dup2 (stdout)");
std::exit(1);
}
if (dup2(output_fd, STDERR_FILENO) < 0) {
std::perror("dup2 (stderr)");
std::exit(1);
}
if (execl(program, program, (char*)nullptr) < 0) {
// These messages will actually go into the file.
std::cerr << "Failed to exec program " << program << ":"
<< std::endl;
std::perror("execl");
std::exit(1);
}
}
return pid;
}
It is possible, for POSIX, because the POSIX shells do this. Executing a program has two steps, for POSIX. First use fork to clone the parent process to create the child process. Then have the child process use one of the exec family of system calls to execute the chosen program instead of the program of the parent. In between those two steps the code executing for the child process can do additional operations, which will affect the environment of the program to be executed. In particular, the code could open a file descriptor to the file to be redirected to, close the stdout file descriptor, then duplicate the file's file descriptor to the value (1) used for stdout.
You could create own pipes and attach them to the child process.
Create 3 pipes. they are going to replace stdin, stdout, stderr of the child.
fork()
In subprocess close() the parent end of the pipes. Close stdin,stdout and stderr.
The parent process close() the child end of the pipes.
dup2() the pipe ends in the child process that are intended to work as the new stdin,out,err
exec() the child.
Now you got all Output from the child to the pipe in the parent. Ofcourse you need to read from the pipes that come from the child or it will block on any write to the stdout/stderr. For this you could use a select(), poll(), epoll() multiplexing algorithm.
See
https://linux.die.net/man/2/pipe
https://linux.die.net/man/2/dup2
https://linux.die.net/man/2/execve
https://linux.die.net/man/2/fork
First off: this is not a Unix/Linux system. I am working on an IBM AS/400 V7R1 (C++ 98) and do not have access to fork(). Nevertheless, I do have spawnp() to start new child processes and the AS/400 supports the notion of process groups.
In my system, I have a "head" program that starts X number of children. This head calls accept() on incoming connections and immediately gives the socket away to one of the child process via sendmsg(). The children are all sitting on recvmsg(). For the head program, it goes something like this:
rc = socketpair(AF_UNIX, SOCK_DGRAM, 0, pair_sd);
if (rc != 0) {
perror("socketpair() failed");
close(listen_sd);
exit(-1);
}
server_sd = pair_sd[0];
worker_sd = pair_sd[1];
// do some other stuff, set up arguments for spawnp()...
// ...
spawn_fdmap[0] = worker_sd;
for (int i = 0; i < numOfChildren; i++) {
pid = spawnp(spawn_argv[0], 1, spawn_fdmap, &inherit, spawn_argv, spawn_envp);
if (pid < 0) {
CERR << "errno=" << errno << ", " << strerror(errno) << endl;
CERR << "command line [";
for (int x = 0; spawn_argv[x] != 0; ++x) {
cerr << spawn_argv[x] << " ";
}
cerr << ']' << endl;
close(listen_sd);
exit(-1);
}
else {
CERR << "Child worker PID = " << pid << endl;
child_pids.push_back(pid);
}
}
// Close down the worker side of the socketpair.
close(worker_sd);
I've got a reason/scheme to start additional child processes after initial program start. I plan to send the head program some signal which would cause the spawnp() call to execute again. The "close(worker_sd)" has me concerned though. Can I call spawnp() again after I've closed the worker socket? It's just a number, after all. Is it OK to keep the worker_sd open?
Can I call spawnp() again after I've closed the worker socket?
After you called close on that socket, the file descriptor is no longer valid in this process.
You probably want a separate socketpair for each child process, so that messages from different child processes do not get interleaved/corrupted.
I think calling socketpair() for every child is unnecessary, and it means having to keep track of additional sockets on the server side. What I found is that removing the close() on 'worker_sd' allows me to create as many additional child processes as I want. Closing it and creating a child process caused the new child to die when it tried to receive something from the parent. I felt this is what would happen, and it did.
Is there a way to execute a binary from my C++ program without a shell? Whenever I use system my command gets run via a shell.
You need to:
fork the process
call one of the "exec" functions in the child process
(if necessary) wait for it to stop
For example, this program runs ls.
#include <iostream>
#include <unistd.h>
#include <sys/wait.h>
// for example, let's "ls"
int ls(const char *dir) {
int pid, status;
// first we fork the process
if (pid = fork()) {
// pid != 0: this is the parent process (i.e. our process)
waitpid(pid, &status, 0); // wait for the child to exit
} else {
/* pid == 0: this is the child process. now let's load the
"ls" program into this process and run it */
const char executable[] = "/bin/ls";
// load it. there are more exec__ functions, try 'man 3 exec'
// execl takes the arguments as parameters. execv takes them as an array
// this is execl though, so:
// exec argv[0] argv[1] end
execl(executable, executable, dir, NULL);
/* exec does not return unless the program couldn't be started.
when the child process stops, the waitpid() above will return.
*/
}
return status; // this is the parent process again.
}
int main() {
std::cout << "ls'ing /" << std::endl;
std::cout << "returned: " << ls("/") << std::endl;
return 0;
}
And the output is:
ls'ing /
bin dev home lib lib64 media opt root sbin srv tmp var
boot etc initrd.img lib32 lost+found mnt proc run selinux sys usr vmlinuz
returned: 0
I used popen, fgets and pclose functions to execute external command line program and redirect its output.
I am launching a command using system api (I am ok with using this api with C/C++). The command I pass may hang at times and hence I would like to kill after certain timeout.
Currently I am using it as:
system("COMMAND");
I want to use it something like this:
Run a command using a system independent API (I don't want to use CreateProcess since it is for Windows only) Kill the process if it does not exit after 'X' Minutes.
Since system() is a platform-specific call, there cannot be a platform-independent way of solving your problem. However, system() is a POSIX call, so if it is supported on any given platform, the rest of the POSIX API should be as well. So, one way to solve your problem is to use fork() and kill().
There is a complication in that system() invokes a shell, which will probably spawn other processes, and I presume you want to kill all of them, so one way to do that is to use a process group. The basic idea is use fork() to create another process, place it in its own process group, and kill that group if it doesn't exit after a certain time.
A simple example - the program forks; the child process sets its own process group to be the same as its process ID, and uses system() to spawn an endless loop. The parent process waits 10 seconds then kills the process group, using the negative value of the child process PID. This will kill the forked process and any children of that process (unless they have changed their process group.)
Since the parent process is in a different group, the kill() has no effect on it.
#include <unistd.h>
#include <stdlib.h>
#include <signal.h>
#include <stdio.h>
int main() {
pid_t pid;
pid = fork();
if(pid == 0) { // child process
setpgid(getpid(), getpid());
system("while true ; do echo xx ; sleep 5; done");
} else { // parent process
sleep(10);
printf("Sleep returned\n");
kill(-pid, SIGKILL);
printf("killed process group %d\n", pid);
}
exit(0);
}
There is no standard, cross-platform system API. The hint is that they are system APIs! We're actually "lucky" that we get system, but we don't get anything other than that.
You could try to find some third-party abstraction.
Check below C++ thread based attempt for linux. (not tested)
#include <iostream>
#include <string>
#include <thread>
#include <stdio.h>
using namespace std;
// execute system command and get output
// http://stackoverflow.com/questions/478898/how-to-execute-a-command-and-get-output-of-command-within-c
std::string exec(const char* cmd) {
FILE* pipe = popen(cmd, "r");
if (!pipe) return "ERROR";
char buffer[128];
std::string result = "";
while(!feof(pipe)) {
if(fgets(buffer, 128, pipe) != NULL)
result += buffer;
}
pclose(pipe);
return result;
}
void system_task(string& cmd){
exec(cmd.c_str());
}
int main(){
// system commad that takes time
string command = "find /";
// run the command in a separate thread
std::thread t1(system_task, std::ref(command));
// gives some time for the system task
std::this_thread::sleep_for(chrono::milliseconds(200));
// get the process id of the system task
string query_command = "pgrep -u $LOGNAME " + command;
string process_id = exec(query_command.c_str());
// kill system task
cout << "killing process " << process_id << "..." << endl;
string kill_command = "kill " + process_id;
exec(kill_command.c_str());
if (t1.joinable())
t1.join();
cout << "continue work on main thread" << endl;
return 0;
}
I had a similar problem, in a Qt/QML development: I want to start a bash command, while continuing to process events on the Qt Loop, and killing the bash command if it takes too long.
I came up with the following class that I'm sharing here (see below), in hope it may be of some use to people with a similar problem.
Instead of calling a 'kill' command, I call a cleanupCommand supplied by the developper. Example: if I'm to call myscript.sh and want to check that it won't last run for more than 10 seconds, I'll call it the following way:
SystemWithTimeout systemWithTimeout("myScript.sh", 10, "killall myScript.sh");
systemWithTimeout.start();
Code:
class SystemWithTimeout {
private:
bool m_childFinished = false ;
QString m_childCommand ;
int m_seconds ;
QString m_cleanupCmd ;
int m_period;
void startChild(void) {
int rc = system(m_childCommand.toUtf8().data());
if (rc != 0) SYSLOG(LOG_NOTICE, "Error SystemWithTimeout startChild: system returned %d", rc);
m_childFinished = true ;
}
public:
SystemWithTimeout(QString cmd, int seconds, QString cleanupCmd)
: m_childFinished {false}, m_childCommand {cmd}, m_seconds {seconds}, m_cleanupCmd {cleanupCmd}
{ m_period = 200; }
void setPeriod(int period) {m_period = period;}
void start(void) ;
};
void SystemWithTimeout::start(void)
{
m_childFinished = false ; // re-arm the boolean for 2nd and later calls to 'start'
qDebug()<<"systemWithTimeout"<<m_childCommand<<m_seconds;
QTime dieTime= QTime::currentTime().addSecs(m_seconds);
std::thread child(&SystemWithTimeout::startChild, this);
child.detach();
while (!m_childFinished && QTime::currentTime() < dieTime)
{
QTime then = QTime::currentTime();
QCoreApplication::processEvents(QEventLoop::AllEvents, m_period); // Process events during up to m_period ms (default: 200ms)
QTime now = QTime::currentTime();
int waitTime = m_period-(then.msecsTo(now)) ;
QThread::msleep(waitTime); // wait for the remaning of the 200 ms before looping again.
}
if (!m_childFinished)
{
SYSLOG(LOG_NOTICE, "Killing command <%s> after timeout reached (%d seconds)", m_childCommand.toUtf8().data(), m_seconds);
int rc = system(m_cleanupCmd.toUtf8().data());
if (rc != 0) SYSLOG(LOG_NOTICE, "Error SystemWithTimeout 164: system returned %d", rc);
m_childFinished = true ;
}
}
I do not know any portable way to do that in C nor C++ languages. As you ask for alternatives, I know it is possible in other languages. For example in Python, it is possible using the subprocess module.
import subprocess
cmd = subprocess.Popen("COMMAND", shell = True)
You can then test if COMMAND has ended with
if cmd.poll() is not None:
# cmd has finished
and you can kill it with :
cmd.terminate()
Even if you prefere to use C language, you should read the documentation for subprocess module because it explains that internally it uses CreateProcess on Windows and os.execvp on Posix systems to start the command, and it uses TerminateProcess on Windows and SIG_TERM on Posix to stop it.
I'm trying to create a parent and a child processes that would communicate through a pipe.
I've setup the child to listen to its parent through a pipe, with a read command running in a while loop.
In order to debug my program I print debug messages to the standard output (note that my read command is set to the pipe with a file descriptor different than 0 or 1).
From some reason these debug messages are being received in the read command of my child process. I can't understand why this is happening. What could be causing this? What elegant solution do I have to solve it (apart from writing to the standard error instead of output)?
This code causes an endless loop because of the cout message that just triggers another read. Why? Notice that the child process exists upon receiving a CHILD_EXIT_CODE signal from parent.
int myPipe[2]
pipe(myPipe);
if(fork() == 0)
{
int readPipe = myPipe[0];
while(true)
{
size_t nBytes = read(readPipe, readBuffer, sizeof(readBuffer));
std::cout << readBuffer << "\n";
int newPosition = atoi(readBuffer);
if(newPosition == CHILD_EXIT_CODE)
{
exit(0);
}
}
}
Edit: Code creating the pipe and fork
I do not know what is doing your parent process (you did not post your code), but because of your description it seems like your parent and child processes are sharing the same stdout stream (the child inherits copies of the parent's set of open file descriptors; see man fork)
I guess, what you should do is to attach stdout and stderr streams in your parent process to the write side of your pipes (you need one more pipe for the stderr stream)
This is what I would try if I were in your situation (in my opinion you are missing dup2):
pid_t pid; /*Child or parent PID.*/
int out[2], err[2]; /*Store pipes file descriptors. Write ends attached to the stdout*/
/*and stderr streams.*/
// Init value as error.
out[0] = out[1] = err[0] = err[1] = -1;
/*Creating pipes, they will be attached to the stderr and stdout streams*/
if (pipe(out) < 0 || pipe(err) < 0) {
/* Error: you should log it */
exit (EXIT_FAILURE);
}
if ((pid=fork()) == -1) {
/* Error: you should log it */
exit (EXIT_FAILURE);
}
if (pid != 0) {
/*Parent process*/
/*Attach stderr and stdout streams to your pipes (their write end)*/
if ((dup2(out[1], 1) < 0) || (dup2(err[1], 2) < 0)) {
/* Error: you should log it */
/* The child is going to be an orphan process you should kill it before calling exit.*/
exit (EXIT_FAILURE);
}
/*WHATEVER YOU DO WITH YOUR PARENT PROCESS*/
/* The child is going to be an orphan process you should kill it before calling exit.*/
exit(EXIT_SUCCESS);
}
else {
/*Child process*/
}
You should not forget a couple of things:
wait or waitpid to release associated memory to child process when it dies. wait or waitpid must be called from parent process.
If you use wait or waitpid you might have to think about blocking SIGCHLD before calling fork and in that case you should unblock SIGCHLD in your child process right after fork, at the beginning of your child process code (A child created via fork(2) inherits a copy of its parent's signal mask; see sigprocmask).
.
Something that many times is forgotten. Be aware of EINTR error. dup2, waitpid/wait, read and many others are affected by this error.
If your parent process dies before your child process you should try to kill the child process if you do not want it to become an orphan one.
Take a look at _exit. Perhaps you should use it in your child process instead of exit.