Linux Pipe replace stdio - issues with MPI - c++

This question is the next step after resolving the issue discussed in:
Piping for input/output
I use pipes to pass a string via stdin to an external program called GULP, and receive the stdout of GULP as input for my program. This works fine on one processor, but on two or more processors there's a problem (let's say it's just 2 cores). The program GULP uses a temporary file and it seems that the two processors launch GULP simultaneously and then GULP tries to perform multiple operations on the same file at the same time (maybe simultaneous writes). GULP reports "error opening file".
I am testing this code on a laptop with multiple cores running Ubuntu, but the code is intended for a distributed-memory HPC (I'm using OpenMPI). Assume for the sake of this discussion that I cannot modify GULP.
I'm hoping that there's some straightforward way to get GULP to create two independent temporary files and continue functioning as normal. Am I asking for too much?
Hopefully this pseudo code will help (assume 2 processors):
int main()
{
MPI_Init(&argc,&argv);
MPI_Comm_rank(…);
MPI_Comm_size(…);
int loopmin, loopmax;//distributes the loop among each processor
for (int i = loopmin; i < loopmax; i++)
{
Launch_GULP(…);//launches external program
}
return 0;
}
Launch_GULP(…)
{
int fd_p2c[2], fd_c2p[2];
pipe(fd_p2c);
pipe(fd_c2p);
childpid = fork();
//the rest follows as in accepted answer in above link
//so i'll highlight the interesting stuff
if (childpid < 0)
{
perror("bad");
exit(-1);
}
else if (childpid == 0)
{
//call dup2, etc
execl( …call the program… );
}
else
{
//the interesting stuff
close(fd_p2c[0]);
close(fd_c2p[1]);
write(fd_p2c[1],…);
close(fd_p2c[1]);
while(1)
{
bytes_read = read(fd_c2p[0],…);//read GULP output
if (bytes_read <= 0)
break;
//pass info to read buffer & append null terminator
}
close(fd_c2p[0]);
if(kill(childpid,SIGTERM) != 0)
{
perror("Failed to kill child… tragic");
exit(1);
}
waitpid(childpid, NULL, 0);
}
//end piping… GULP has reported an error via stdout
//that error is stored in the buffer string
//consequently an error is triggered in my code and the program exits
}

Related

Simplest IPC from one Linux app to another in C++ on raspberry pi

I need the simplest most reliable IPC method from one C++ app running on the RPi to another app.
All I'm trying to do is send a string message of 40 characters from one app to another
The first app is running as a service on boot, the other app is started at a later time and is frequently exited and restarted for debugging
The frequent debugging for the second app is whats causing problems with the IPCs I've tried so far
I've tried about 3 different methods and here is where they failed:
File FIFO, the problem is one program hangs while the other program is writing to the file
Shared memory: cannot initialize on one thread and read from another thread. Also frequent exiting while debugging causing GDB crashes with the following GDB command is taking too long to complete -stack-list-frames --thread 1
UDP socket with localhost - same issue as above, plus improper exits block the socket, forcing me to reboot device
Non blocking pipe - not getting any messages on the receiving process
What else can I try? I dont want to get the DBus library, seems too complex for this application.
Any simple server and client code or a link to it would be helpful
Here is my non-blockign pipe code, that doesnt work for me,
I assume its because I dont have a reference to the pipe from one app to the other
Code sourced from here: https://www.geeksforgeeks.org/non-blocking-io-with-pipes-in-c/
char* msg1 = "hello";
char* msg2 = "bye !!";
int p[2], i;
bool InitClient()
{
// error checking for pipe
if(pipe(p) < 0)
exit(1);
// error checking for fcntl
if(fcntl(p[0], F_SETFL, O_NONBLOCK) < 0)
exit(2);
//Read
int nread;
char buf[MSGSIZE];
// write link
close(p[1]);
while (1) {
// read call if return -1 then pipe is
// empty because of fcntl
nread = read(p[0], buf, MSGSIZE);
switch (nread) {
case -1:
// case -1 means pipe is empty and errono
// set EAGAIN
if(errno == EAGAIN) {
printf("(pipe empty)\n");
sleep(1);
break;
}
default:
// text read
// by default return no. of bytes
// which read call read at that time
printf("MSG = % s\n", buf);
}
}
return true;
}
bool InitServer()
{
// error checking for pipe
if(pipe(p) < 0)
exit(1);
// error checking for fcntl
if(fcntl(p[0], F_SETFL, O_NONBLOCK) < 0)
exit(2);
//Write
// read link
close(p[0]);
// write 3 times "hello" in 3 second interval
for(i = 0 ; i < 3000000000 ; i++) {
write(p[0], msg1, MSGSIZE);
sleep(3);
}
// write "bye" one times
write(p[0], msg2, MSGSIZE);
return true;
}
Please consider ZeroMQ
https://zeromq.org/
It is lightweight and has wrapper for all major programming languages.

Runtime issues on ARM but runs fine on x86

I recently created an application that I developed on my x86 Linux machine. Basically it's just two threads that communicate over a pipe() with each other. Thread 0 listens on the read end and Thread 1 writes into that pipe. That program worked perfectly fine.
But when I copied the sources over to a RaspberryPi and built it, there were some runtime issues (but compiled with no errors). It seems that thread0 never gets something out of the pipe, it just blocks.
Since pipes are made for Interprocess communication, i thought it would also be thread safe (since there also are two different file descriptors for read and write end).
BUT: Stepping through the program in the Qt Creator debugger on the RPi, everything seemed to work fine! I know the debugger initializing certain variables different can lead to such conditions, but I couldn't find any usages of uninitialized variables etc. in my Code.
thread 1:
void *midiThread(void *fds)
{
midiDevice = ((int*)fds)[0]; // device file for midi input
midiBuffer = ((int*)fds)[1]; // write end of the pipe
unsigned char rawBuffer[MIDI_MSG_LENGTH];
while (read(midiDevice, rawBuffer, MIDI_MSG_LENGTH)
>= MIDI_MSG_LENGTH)
{
struct midievent_t currentEvent;
unsigned char *rawBuffer = (unsigned char *)buffer;
currentEvent.channel = rawBuffer[0] & 0x0f;
// ....
write(midiBuffer, &currentEvent, sizeof(struct midievent_t));
}
close(midiBuffer);
return NULL;
}
main thread:
void MidiInput::createMidiThread()
{
if (pipe(_midiBufferPipe) < 0)
{
// error
}
int fds[2];
fds[0] = _midiFileDescriptor;
fds[1] = _midiBufferPipe[1];
pthread_create(&_midiThreadId, NULL,
midiThread, fds);
}
bool MidiInput::read(midievent_t *event)
{
if (!_initialized)
{
return false;
}
if (read(_midiBufferPipe[0], event, sizeof(struct midievent_t))
< sizeof(struct midievent_t))
{
// some error
return _initialized = false;
}
return true;
}

Read Write issues with Pseudo Terminal in Linux

I am writing a C++ program that would interact with an external process. The external process is written in C# and runs on mono. Note that I cannot modify the C# code as it is not a program written by me.
In this regard, I first set out by using pipes, which of course as I later realized is fully buffered and hence I faced a lot of sync issues. Essentially the external process had to flush its output after every write and this was not possible.
The next thing that I was about to try out was files, but however I found out that using pseudo-terminals would be more apt in my case. Here is some sample code that I have written:
int main()
{
int fdm, fds, rc, pid;
bool rValue;
/* Setup Master pty*/
rValue = rValue && (fdm = posix_openpt(O_RDWR)) >= 0 &&
(rc = grantpt(fdm)) == 0 && (rc = unlockpt(fdm) == 0);
if (rValue) {
/* Open Slave pty */
fds = open(ptsname(fdm), O_RDWR);
pid = fork();
if(pid < 0)
perror("fork failed");
else if(pid == 0) //child
{
close(fdm); //close master
struct termios slave_orig_term_settings;
struct termios new_term_settings;
tcgetattr(slaveTTY, &slave_orig_term_settings);
new_term_settings = slave_orig_term_settings;
cfmakeraw(&new_term_settings);
tcsetattr(slaveTTY, TCSANOW, &new_term_settings);
//redirect I/O of this process
close(0);
close(1);
close(2);
dup(slaveTTY);
dup(slaveTTY);
dup(slaveTTY);
close(slaveTTY);
setsid();
ioctl(0, TIOCSCTTY, 1);
//launch the external process and replace its image in this process
execve(argv[0],...);
}
else
{
close(fds); //close slave
//Perform some interaction
write(something using fdm);
//Assume fdsets declared and set somewhere here
select(fdm +1,&fdset,NULL,NULL,NULL);
int readBytes = read(someting using fds);
}
}
return EXIT_SUCCESS;
}
Assume that the fdset and fdclr for select are being taken care of.
The following issues are being observed in the parent process:
Sometimes read returns with readBytes > 0 but there is nothing present in the buffer
Sometimes whatever has been written to the terminal is read back
Some garbage values such as ^]]49]1R are being dumped on the terminal (this is the actual terminal i.e. my output window)
P.S: When the external process is written in C/C++, this issue is not occuring. Only when I run a C# program in mono.
I think pexpect in python is a good choice if you don't have to do that in C++, it will save you a lot of time. And also you can use python freeze tools like pyinstaller to convert your python script to standalone binary.

waitpid/wexitstatus returning 0 instead of correct return code

I have the helper function below, used to execute a command and get the return value on posix systems. I used to use popen, but it is impossible to get the return code of an application with popen if it runs and exits before popen/pclose gets a chance to do its work.
The following helper function creates a process fork, uses execvp to run the desired external process, and then the parent uses waitpid to get the return code. I'm seeing odd cases where it's refusing to run.
When called with wait = true, waitpid should return the exit code of the application no matter what. However, I'm seeing stdout output that specifies the return code should be non-zero, yet the return code is zero. Testing the external process in a regular shell, then echoing $? returns non-zero, so it's not a problem w/ the external process not returning the right code. If it's of any help, the external process being run is mount(8) (yes, I know I can use mount(2) but that's besides the point).
I apologize in advance for a code dump. Most of it is debugging/logging:
inline int ForkAndRun(const std::string &command, const std::vector<std::string> &args, bool wait = false, std::string *output = NULL)
{
std::string debug;
std::vector<char*> argv;
for(size_t i = 0; i < args.size(); ++i)
{
argv.push_back(const_cast<char*>(args[i].c_str()));
debug += "\"";
debug += args[i];
debug += "\" ";
}
argv.push_back((char*)NULL);
neosmart::logger.Debug("Executing %s", debug.c_str());
int pipefd[2];
if (pipe(pipefd) != 0)
{
neosmart::logger.Error("Failed to create pipe descriptor when trying to launch %s", debug.c_str());
return EXIT_FAILURE;
}
pid_t pid = fork();
if (pid == 0)
{
close(pipefd[STDIN_FILENO]); //child isn't going to be reading
dup2(pipefd[STDOUT_FILENO], STDOUT_FILENO);
close(pipefd[STDOUT_FILENO]); //now that it's been dup2'd
dup2(pipefd[STDOUT_FILENO], STDERR_FILENO);
if (execvp(command.c_str(), &argv[0]) != 0)
{
exit(EXIT_FAILURE);
}
return 0;
}
else if (pid < 0)
{
neosmart::logger.Error("Failed to fork when trying to launch %s", debug.c_str());
return EXIT_FAILURE;
}
else
{
close(pipefd[STDOUT_FILENO]);
int exitCode = 0;
if (wait)
{
waitpid(pid, &exitCode, wait ? __WALL : (WNOHANG | WUNTRACED));
std::string result;
char buffer[128];
ssize_t bytesRead;
while ((bytesRead = read(pipefd[STDIN_FILENO], buffer, sizeof(buffer)-1)) != 0)
{
buffer[bytesRead] = '\0';
result += buffer;
}
if (wait)
{
if ((WIFEXITED(exitCode)) == 0)
{
neosmart::logger.Error("Failed to run command %s", debug.c_str());
neosmart::logger.Info("Output:\n%s", result.c_str());
}
else
{
neosmart::logger.Debug("Output:\n%s", result.c_str());
exitCode = WEXITSTATUS(exitCode);
if (exitCode != 0)
{
neosmart::logger.Info("Return code %d", (exitCode));
}
}
}
if (output)
{
result.swap(*output);
}
}
close(pipefd[STDIN_FILENO]);
return exitCode;
}
}
Note that the command is run OK with the correct parameters, the function proceeds without any problems, and WIFEXITED returns TRUE. However, WEXITSTATUS returns 0, when it should be returning something else.
Probably isn't your main issue, but I think I see a small problem. In your child process, you have...
dup2(pipefd[STDOUT_FILENO], STDOUT_FILENO);
close(pipefd[STDOUT_FILENO]); //now that it's been dup2'd
dup2(pipefd[STDOUT_FILENO], STDERR_FILENO); //but wait, this pipe is closed!
But I think what you want is:
dup2(pipefd[STDOUT_FILENO], STDOUT_FILENO);
dup2(pipefd[STDOUT_FILENO], STDERR_FILENO);
close(pipefd[STDOUT_FILENO]); //now that it's been dup2'd for both, can close
I don't have much experience with forks and pipes in Linux, but I did write a similar function pretty recently. You can take a look at the code to compare, if you'd like. I know that my function works.
execAndRedirect.cpp
I'm using the mongoose library, and grepping my code for SIGCHLD revealed that using mg_start from mongoose results in setting SIGCHLD to SIG_IGN.
From the waitpid man page, on Linux a SIGCHLD set to SIG_IGN will not create a zombie process, so waitpid will fail if the process has already successfully run and exited - but will run OK if it hasn't yet. This was the cause of the sporadic failure of my code.
Simply re-setting SIGCHLD after calling mg_start to a void function that does absolutely nothing was enough to keep the zombie records from being immediately erased.
Per #Geoff_Montee's advice, there was a bug in my redirect of STDERR, but this was not responsible for the problem as execvp does not store the return value in STDERR or even STDOUT, but rather in the kernel object associated with the parent process (the zombie record).
#jilles' warning about non-contiguity of vector in C++ does not apply for C++03 and up (only valid for C++98, though in practice, most C++98 compilers did use contiguous storage, anyway) and was not related to this issue. However, the advice on reading from the pipe before blocking and checking the output of waitpid is spot-on.
I've found that pclose does NOT block and wait for the process to end, contrary to the documentation (this is on CentOS 6). I've found that I need to call pclose and then call waitpid(pid,&status,0); to get the true return value.

How to run a C++, PortAudio application on startup on Angstrom Linux on a BeagleBoard?

I have a command-line application called xooky_nabox that was programmed using c++. It reads a puredata patch, processes signals from the audio in jack of a beagleboard and outputs signals through the audio out jack.
I want the application to run wen the beagleoard starts up and stay running until the board is shut down. There is no GUI and no keyboard or monitor attached to it, just the audio in and out jacks.
If I run the application manually everything works fine:
xooky_nabox -audioindev 1 -audiooutdev 1 /var/xooky/patch.pd
And it also runs fine if I run it in the background:
xooky_nabox -audioindev 1 -audiooutdev 1 /var/xooky/patch.pd &
Now, let me show the code layout of two versions of the program (The full thing is at https://github.com/rvega/XookyNabox):
Version 1, main thread is kept alive:
void sighandler(int signum){
time_t rawtime;
time(&rawtime);
std::ofstream myfile;
myfile.open ("log.txt",std::ios::app);
myfile << ctime(&rawtime) << " Caught signal:" << signum << " " << strsignal(signum) << "\n";
myfile.close();
if(signum == 15 || signum == 2){
exit(0);
}
}
int main (int argc, char *argv[]) {
// Subscribe to all system signals for debugging purposes.
for(int i=0; i<64; i++){
signal(i, sighandler);
}
// Sanity checks, error and help messages, etc.
parseParameters(argc, argv);
//Start Signal processing and Audio
initZenGarden();
initAudioIO();
// Keep the program alive.
while(1){
sleep(10);
}
// This is obviously never reached, so far no problems with that...
stopAudioIO();
stopZengarden();
return 0;
}
static int paCallback( const void *inputBuffer, void *outputBuffer, unsigned long framesPerBuffer, const PaStreamCallbackTimeInfo* timeInfo, PaStreamCallbackFlags statusFlags, void *userData ){
// This is called by PortAudio when the output buffer is about to run dry.
}
Version 2, execution is forked and detached from the terminal that launched it:
void go_daemon(){
// Run the program as a daemon.
pid_t pid, sid;
pid = fork(); // Fork off the parent process
if (pid < 0) {
exit(EXIT_FAILURE);
}
if (pid > 0) {
exit(EXIT_SUCCESS); // If child process started ok, exit the parent process
}
umask(0); // Change file mode mask
sid = setsid(); // Create a new session ID for the child process
if (sid < 0) {
// TODO: Log failure
exit(EXIT_FAILURE);
}
if((chdir("/")) < 0){ //Change the working directory to "/"
//TODO: Log failre
exit(EXIT_FAILURE);
}
close(STDIN_FILENO);
close(STDOUT_FILENO);
close(STDERR_FILENO);
}
int main (int argc, char *argv[]) {
go_daemon();
// Subscribe to all system signals for debugging purposes.
for(int i=0; i<64; i++){
signal(i, sighandler);
}
// Sanity checks, error and help messages, etc.
parseParameters(argc, argv);
//Start Signal processing and Audio
initZenGarden();
initAudioIO();
// Keep the program alive.
while(1){
sleep(10);
}
// This is obviously never reached, so far no problems with that...
stopAudioIO();
stopZengarden();
return 0;
}
Trying to run it at startup
I've tried running both versions of the program at startup using a few methods. The outcome is always the same. When the beagle starts up, I can hear sound beign output for a fraction of a second, the sound then stops and the login screen is presented (I have a serial terminal attached to the board and minicom running on my computer). The weirdest thing to me is that the xooky_nabox process is actually kept running after login but there is no sound output...
Here's what I've tried:
Adding a #reboot entry to crontab and launching the program with a trailing ampersand (version 1 of the program):
#reboot xooky_nabox <params> &
Added a start-stop-daemon to crontab (version 1):
#reboot start-stop-daemon -S -b --user daemon -n xooky_nabox -a /usr/bin/xooky_nabox -- <params>
Created a script at /etc/init.d/xooky and did
$chmod +x xooky
$update-rc.d xooky defaults
And tried different versions of the startup script: start-stop-daemon with version 1, calling the program directly with a trailing ampersand (version 1), calling the program directly with no trailing ampersand (version 2).
Also, if I run the program manually from the serial terminal or from a ssh session (usb networking); and then I run top, the program will run fine for a few seconds consuming around 15% cpu. It will then stop outputing sound, and it's cpu consumption will rise to around 30%. My log.txt file shows no signal sent to the program by the OS in this scenario.
When version 2 of the program is ran at startup, the log wil show something like:
Mon Jun 6 02:44:49 2011 Caught signal:18 Continued
Mon Jun 6 02:44:49 2011 Caught signal:15 Terminated
Does anyone have any ideas on how to debug this? Suggestions on how to launch my program at startup?
In version 2,
I think you should open (and dup2) /dev/null to STDIN/STDOUT/STDERR. Just closing the handle would cause problem.
something like this:
int fd = open("/dev/null", O_RDWR);
dup2( fd, STDOUT_FILENO );
(I have no idea what start-stop-daemon do. Can't help version 1, sorry)
There is C function to create a daemon
#include <unistd.h>
int daemon(int nochdir, int noclose);
More information can be found in man pages for daemon(3)
Maybe it will help.
And if you want to launch you daemon when you linux start, you should find out which init version you are using in you distro, but usually, you can just add command to execute you daemon to /etc/init.d/rc (but it seems to be no so good idea). This file is executed by init when linux is starting.
I ended up ditching PortAudio and implementing a JACK client which runs it's own server so this issue was not relevant for me anymore.