I'm working on a server application that's going to work on Linux and Mac OS X. It goes like this:
start main application
fork of the controller process
call lock_down() in the controller process
terminate main application
the controller process then forks again, creating a worker process
eventually the controller keeps forking more worker processes
I can log using several of methods (e.g. syslog or a file) but right now I'm pondering about syslog. The "funny" thing is that no syslog output is ever seen in the controller process unless I include the #ifdef section below.
The worker processes logs flawlessly in Mac OS X and linux with or without the ifdef'ed section below. The controller also logs flawlessly in Mac OS X without the #ifdef'ed section, but on linux the ifdef is needed if I want to see any output into syslog (or the log file for that matter) from the controller process.
So, why is that?
static int
lock_down(void)
{
struct rlimit rl;
unsigned int n;
int fd0;
int fd1;
int fd2;
// Reset file mode mask
umask(0);
// change the working directory
if ((chdir("/")) < 0)
return EXIT_FAILURE;
// close any and all open file descriptors
if (getrlimit(RLIMIT_NOFILE, &rl))
return EXIT_FAILURE;
if (RLIM_INFINITY == rl.rlim_max)
rl.rlim_max = 1024;
for (n = 0; n < rl.rlim_max; n++) {
#ifdef __linux__
if (3 == n) // deep magic...
continue;
#endif
if (close(n) && (EBADF != errno))
return EXIT_FAILURE;
}
// attach file descriptors 0, 1 and 2 to /dev/null
fd0 = open("/dev/null", O_RDWR);
fd1 = dup2(fd0, 1);
fd2 = dup2(fd0, 2);
if (0 != fd0)
return EXIT_FAILURE;
return EXIT_SUCCESS;
}
camh was close, but using closelog() was the idea that did the trick so the honor goes to jilles. Something else, aside from closing a file descriptor from under syslogs feet must go on though. To make the code work I added a call to closelog() just before the loop:
closelog();
for (n = 0; n < rl.rlim_max; n++) {
if (close(n) && (EBADF != errno))
return EXIT_FAILURE;
}
I was relying on a verbatim understanding of the manual page, saying:
The use of openlog() is optional; it will automatically be called by syslog() if necessary...
I interpreted this as saying that syslog would detect if the file descriptor was closed under it. Apparently it did not. An explicit closelog() on linux was needed to tell syslog that the descriptor was closed.
One more thing that still perplexes me is that not using closelog() prevented the first forked process (the controller) from even opening and using a log file. The following forked processes could use syslog or a log file with no problems. Maybe there are some caching effect in the filesystem that make the first forked process having an unreliable "idea" of which file descriptors are available, while the next set of forked process are sufficiently delayed to not be affected by this?
The special aspect of file descriptor 3 is that it will usually be the first file descriptor returned from a system call that allocates a new file descriptor, given that 0, 1 and 2 are usually set up for stdin, stdout and stderr.
This means that if any library function you have called allocates a file descriptor for its own internal purposes in order to perform its functions, it will get fd 3.
The openlog(3) library call will need to open /dev/log to communicate with the syslog daemon. If you subsequently close all file descriptors, you may break the syslog library functions if they are not written in a way to handle that.
The way to debug this on Linux is to use strace to trace the actual system calls that are being made; the use of a file descriptor for syslog then becomes obvious:
$ cat syslog_test.c
#include <stdio.h>
#include <syslog.h>
int main(void)
{
openlog("test", LOG_PID, LOG_LOCAL0);
syslog(LOG_ERR, "waaaaaah");
closelog();
return 0;
}
$ gcc -W -Wall -o syslog_test syslog_test.c
$ strace ./syslog_test
...
socket(PF_FILE, SOCK_DGRAM, 0) = 3
fcntl64(3, F_SETFD, FD_CLOEXEC) = 0
connect(3, {sa_family=AF_FILE, path="/dev/log"}, 16) = 0
send(3, "<131>Aug 21 00:47:52 test[24264]"..., 42, MSG_NOSIGNAL) = 42
close(3) = 0
exit_group(0) = ?
Process 24264 detached
syslog(3) may keep a file descriptor to syslogd's socket open; closing this under its feet is likely to cause problems. A closelog(3) call may help.
Syslog binds on a given descriptor at startup. Most of the time descriptor 3. If you close it no logs.
syslog-ng -d -v
Gives you more info about what it's doing behind the scenes.
The output should look like something like this:
binding fd 3, inetaddr: 0.0.0.0, port: 514
io.c: Preparing fd 3 for reading
io.c: Preparing fd 4 for reading
binding fd 5, unixaddr: /dev/log
io.c: listening on fd 5
Related
I need the simplest most reliable IPC method from one C++ app running on the RPi to another app.
All I'm trying to do is send a string message of 40 characters from one app to another
The first app is running as a service on boot, the other app is started at a later time and is frequently exited and restarted for debugging
The frequent debugging for the second app is whats causing problems with the IPCs I've tried so far
I've tried about 3 different methods and here is where they failed:
File FIFO, the problem is one program hangs while the other program is writing to the file
Shared memory: cannot initialize on one thread and read from another thread. Also frequent exiting while debugging causing GDB crashes with the following GDB command is taking too long to complete -stack-list-frames --thread 1
UDP socket with localhost - same issue as above, plus improper exits block the socket, forcing me to reboot device
Non blocking pipe - not getting any messages on the receiving process
What else can I try? I dont want to get the DBus library, seems too complex for this application.
Any simple server and client code or a link to it would be helpful
Here is my non-blockign pipe code, that doesnt work for me,
I assume its because I dont have a reference to the pipe from one app to the other
Code sourced from here: https://www.geeksforgeeks.org/non-blocking-io-with-pipes-in-c/
char* msg1 = "hello";
char* msg2 = "bye !!";
int p[2], i;
bool InitClient()
{
// error checking for pipe
if(pipe(p) < 0)
exit(1);
// error checking for fcntl
if(fcntl(p[0], F_SETFL, O_NONBLOCK) < 0)
exit(2);
//Read
int nread;
char buf[MSGSIZE];
// write link
close(p[1]);
while (1) {
// read call if return -1 then pipe is
// empty because of fcntl
nread = read(p[0], buf, MSGSIZE);
switch (nread) {
case -1:
// case -1 means pipe is empty and errono
// set EAGAIN
if(errno == EAGAIN) {
printf("(pipe empty)\n");
sleep(1);
break;
}
default:
// text read
// by default return no. of bytes
// which read call read at that time
printf("MSG = % s\n", buf);
}
}
return true;
}
bool InitServer()
{
// error checking for pipe
if(pipe(p) < 0)
exit(1);
// error checking for fcntl
if(fcntl(p[0], F_SETFL, O_NONBLOCK) < 0)
exit(2);
//Write
// read link
close(p[0]);
// write 3 times "hello" in 3 second interval
for(i = 0 ; i < 3000000000 ; i++) {
write(p[0], msg1, MSGSIZE);
sleep(3);
}
// write "bye" one times
write(p[0], msg2, MSGSIZE);
return true;
}
Please consider ZeroMQ
https://zeromq.org/
It is lightweight and has wrapper for all major programming languages.
I want to get a Linux command's output string as well as command output status in a C++ program. I am executing Linux commands in my application.
for example:
Command:
rmdir abcd
Command output string:
rmdir: failed to remove `abcd': No such file or directory
Command Status:
1 (Which means command has been failed)
I tried using Linux function system() which gives the output status, and function popen() which gives me output string of a command, but neither function gives me both
the output string and output status of a Linux command.
The output string is in standard output or standard error descriptor (1 or 2, respectively).
You have to redirect these streams (take a look at dup and dup2 function) to a place, where you can read them (for example - a POSIX pipe).
In C I'd do something like this:
int pd[2];
int retValue;
char buffer[MAXBUF] = {0};
pipe(pd);
dup2(pd[1],1);
retValue = system("your command");
read(pd[0], buffer, MAXBUF);
Now, you have (a part of) your output in buffer and the return code in retValue.
Alternatively, you can use a function from exec (i.e. execve) and get the return value with wait or waitpid.
Update: this will redirect only standard output. To redirect standard error, use dup2(pd[1],1).
The simplest solution is to use system, and to redirect standard out and standard error to a temporarly file, which you can delete later.
Unfortunately there's no easy and simple way in C on Linux to do this. Here's an example how to read/write stdout/stderr/stdin of child process correctly.
And when you want to receive exit code you have to use waitpid (complete example is provided on the bottom of the provided page):
endID = waitpid(childID, &status, WNOHANG|WUNTRACED);
Now you just have to join those two together :)
There's also a great free book named Advanced Linux Programming (ALP) containing detailed information about these kinds of problem available here.
Building on Piotr Zierhoffer answer above, here's a function that does just that, and also restores stdout and stderr their original state.
// Execute command <cmd>, put its output (stdout and stderr) in <output>,
// and return its status
int exec_command(string& cmd, string& output) {
// Save original stdout and stderr to enable restoring
int org_stdout = dup(1);
int org_stderr = dup(2);
int pd[2];
pipe(pd);
// Make the read-end of the pipe non blocking, so if the command being
// executed has no output the read() call won't get stuck
int flags = fcntl(pd[0], F_GETFL);
flags |= O_NONBLOCK;
if(fcntl(pd[0], F_SETFL, flags) == -1) {
throw string("fcntl() failed");
}
// Redirect stdout and stderr to the write-end of the pipe
dup2(pd[1], 1);
dup2(pd[1], 2);
int status = system(cmd.c_str());
int buf_size = 1000;
char buf[buf_size];
// Read from read-end of the pipe
long num_bytes = read(pd[0], buf, buf_size);
if(num_bytes > 0) {
output.clear();
output.append(buf, num_bytes);
}
// Restore stdout and stderr and release the org* descriptors
dup2(org_stdout, 1);
dup2(org_stderr, 2);
close(org_stdout);
close(org_stderr);
return status;
}
you can use popen system call, it will redirect output to a file and from file you can redirect output to a string. like :
char buffer[MAXBUF] = {0};
FILE *fd = popen("openssl version -v", "r");
if (NULL == fd)
{
printf("Error in popen");
return;
}
fread(buffer, MAXBUF, 1, fd);
printf("%s",buffer);
pclose(fd);
For more information read man page for popen.
I need to run an external program from within a c++ application. I need the output from that program (i want to see it while the program is still running) and it also needs to get input.
What is the best and most elegant way to redirect the IO? Should it be running in it's own thread? Any examples?
It's running on OSX.
I implemented it like this:
ProgramHandler::ProgramHandler(std::string prog): program(prog){
// Create two pipes
std::cout << "Created Class\n";
pipe(pipe1);
pipe(pipe2);
int id = fork();
std::cout << "id: " << id << std::endl;
if (id == 0)
{
// In child
// Close current `stdin` and `stdout` file handles
close(fileno(stdin));
close(fileno(stdout));
// Duplicate pipes as new `stdin` and `stdout`
dup2(pipe1[0], fileno(stdin));
dup2(pipe2[1], fileno(stdout));
// We don't need the other ends of the pipes, so close them
close(pipe1[1]);
close(pipe2[0]);
// Run the external program
execl("/bin/ls", "bin/ls");
char buffer[30];
while (read(pipe1[0], buffer, 30)) {
std::cout << "Buf: " << buffer << std::endl;
}
}
else
{
// We don't need the read-end of the first pipe (the childs `stdin`)
// or the write-end of the second pipe (the childs `stdout`)
close(pipe1[0]);
close(pipe2[1]);
// Now you can write to `pipe1[1]` and it will end up as `stdin` in the child
// Read from `pipe2[0]` to read from the childs `stdout`
}
}
but as an output i get this:
Created Class
id: 84369
id: 0
I don't understand why it s called twice and why it wont fork the first time. What am I doing/understanding wrong.
If using a POSIX system (like OSX or Linux) then you have to learn the system calls pipe, fork, close, dup2 and exec.
What you do is create two pipes, one for reading from the external application and one for writing. Then you fork to create a new process, and in the child process you set up the pipes as stdin and stdout and then call exec which replaces the child process with an external program using your new stdin and stdout file handles. In the parent process you can not read the output from the child process, and write to its input.
In pseudo-code:
// Create two pipes
pipe(pipe1);
pipe(pipe2);
if (fork() == 0)
{
// In child
// Close current `stdin` and `stdout` file handles
close(FILENO_STDIN);
close(FILENO_STDOUT);
// Duplicate pipes as new `stdin` and `stdout`
dup2(pipe1[0], FILENO_STDIN);
dup2(pipe2[1], FILENO_STDOUT);
// We don't need the other ends of the pipes, so close them
close(pipe1[1]);
close(pipe2[0]);
// Run the external program
exec("/some/program", ...);
}
else
{
// We don't need the read-end of the first pipe (the childs `stdin`)
// or the write-end of the second pipe (the childs `stdout`)
close(pipe1[0]);
close(pipe2[1]);
// Now you can write to `pipe1[1]` and it will end up as `stdin` in the child
// Read from `pipe2[0]` to read from the childs `stdout`
}
Read the manual pages of the system calls for more information about them. You also need to add error checking as all of these system calls may fail.
Well there is a pretty standard way to do this. In general you would like to fork the process and to close the standard I/O (fd 0,1) of the child. Before forking have create two pipes, after forking close the standard input and output in the child and connect them to the pipe, using dup.
Pseudo code, shows only one side of the connection, I'm sure you can figure out the other side.
int main(){
int fd[2]; // file descriptors
pipe(fd);
// Fork child process
if (fork() == 0){
char buffer [80];
close(1);
dup(fd[1]); // this will take the first free discriptor, the one you just closed.
close(fd[1]); // clean up
}else{
close(0);
dup(fd[0]);
close(fd[0]);
}
return 0;
}
After you have the pipe set up and one of the parent threads waiting on a select or something, you can call exec for your external tool and have all the data flowing.
The basic approach to communicate with a different program on POSIX systems is to setup a pipe(), then fork() your program, close() and dup() file descriptors into the correct location, and finally to exec??() the desired executable.
Once this is done, you have your two programs connected with suitable streams. Unfortunately, this doesn't deal with any form of asynchronous processing of the two programs. That is, it is likely that you either want to access the created file descriptor with suitable asynchronous and non-blocking operations (i.e., setup the various file descriptors to be non-blocking and/or access them only when poll() yields results indicating that you can access them). If there is just that one executable it may be easier to control it from a separate thread, though.
A different approach (and if you are also writing the external program) is to use shared memory. Something along the lines of (pseudo code)
// create shared memory
int l_shmid = shmget(key, size ,0600 | IPC_CREAT);
if(l_shmid < 0)
ERROR
// attach to shared memory
dataptr* ptr = (dataptr*)shmat(l_shmid, NULL, 0600);
// run external program
pid_t l_pid = fork();
if(l_pid == (pid_t)-1)
{
ERROR
// detach & delete shared mem
shmdt(ptr);
shmctl(l_shmid,
IPC_RMID,
(shmid_ds *)NULL);
return;
}
else if(l_pid == 0)
{
// child:
execl(path,
args,
NULL);
return;
}
// wait for the external program to finish
int l_stat(0);
waitpid(l_pid, &l_stat, 0);
// read from shmem
memset(mydata, ..,..);
memcpy(mydata, ptr, ...);
// detach & close shared mem
shmdt(ptr);
shmctl(l_shmid,
IPC_RMID,
(shmid_ds *)NULL);
Your external program can write to shared memory in a similar way. No need for pipes & reading/writing/selecting etc.
I am writing a C++ program that would interact with an external process. The external process is written in C# and runs on mono. Note that I cannot modify the C# code as it is not a program written by me.
In this regard, I first set out by using pipes, which of course as I later realized is fully buffered and hence I faced a lot of sync issues. Essentially the external process had to flush its output after every write and this was not possible.
The next thing that I was about to try out was files, but however I found out that using pseudo-terminals would be more apt in my case. Here is some sample code that I have written:
int main()
{
int fdm, fds, rc, pid;
bool rValue;
/* Setup Master pty*/
rValue = rValue && (fdm = posix_openpt(O_RDWR)) >= 0 &&
(rc = grantpt(fdm)) == 0 && (rc = unlockpt(fdm) == 0);
if (rValue) {
/* Open Slave pty */
fds = open(ptsname(fdm), O_RDWR);
pid = fork();
if(pid < 0)
perror("fork failed");
else if(pid == 0) //child
{
close(fdm); //close master
struct termios slave_orig_term_settings;
struct termios new_term_settings;
tcgetattr(slaveTTY, &slave_orig_term_settings);
new_term_settings = slave_orig_term_settings;
cfmakeraw(&new_term_settings);
tcsetattr(slaveTTY, TCSANOW, &new_term_settings);
//redirect I/O of this process
close(0);
close(1);
close(2);
dup(slaveTTY);
dup(slaveTTY);
dup(slaveTTY);
close(slaveTTY);
setsid();
ioctl(0, TIOCSCTTY, 1);
//launch the external process and replace its image in this process
execve(argv[0],...);
}
else
{
close(fds); //close slave
//Perform some interaction
write(something using fdm);
//Assume fdsets declared and set somewhere here
select(fdm +1,&fdset,NULL,NULL,NULL);
int readBytes = read(someting using fds);
}
}
return EXIT_SUCCESS;
}
Assume that the fdset and fdclr for select are being taken care of.
The following issues are being observed in the parent process:
Sometimes read returns with readBytes > 0 but there is nothing present in the buffer
Sometimes whatever has been written to the terminal is read back
Some garbage values such as ^]]49]1R are being dumped on the terminal (this is the actual terminal i.e. my output window)
P.S: When the external process is written in C/C++, this issue is not occuring. Only when I run a C# program in mono.
I think pexpect in python is a good choice if you don't have to do that in C++, it will save you a lot of time. And also you can use python freeze tools like pyinstaller to convert your python script to standalone binary.
I have a command-line application called xooky_nabox that was programmed using c++. It reads a puredata patch, processes signals from the audio in jack of a beagleboard and outputs signals through the audio out jack.
I want the application to run wen the beagleoard starts up and stay running until the board is shut down. There is no GUI and no keyboard or monitor attached to it, just the audio in and out jacks.
If I run the application manually everything works fine:
xooky_nabox -audioindev 1 -audiooutdev 1 /var/xooky/patch.pd
And it also runs fine if I run it in the background:
xooky_nabox -audioindev 1 -audiooutdev 1 /var/xooky/patch.pd &
Now, let me show the code layout of two versions of the program (The full thing is at https://github.com/rvega/XookyNabox):
Version 1, main thread is kept alive:
void sighandler(int signum){
time_t rawtime;
time(&rawtime);
std::ofstream myfile;
myfile.open ("log.txt",std::ios::app);
myfile << ctime(&rawtime) << " Caught signal:" << signum << " " << strsignal(signum) << "\n";
myfile.close();
if(signum == 15 || signum == 2){
exit(0);
}
}
int main (int argc, char *argv[]) {
// Subscribe to all system signals for debugging purposes.
for(int i=0; i<64; i++){
signal(i, sighandler);
}
// Sanity checks, error and help messages, etc.
parseParameters(argc, argv);
//Start Signal processing and Audio
initZenGarden();
initAudioIO();
// Keep the program alive.
while(1){
sleep(10);
}
// This is obviously never reached, so far no problems with that...
stopAudioIO();
stopZengarden();
return 0;
}
static int paCallback( const void *inputBuffer, void *outputBuffer, unsigned long framesPerBuffer, const PaStreamCallbackTimeInfo* timeInfo, PaStreamCallbackFlags statusFlags, void *userData ){
// This is called by PortAudio when the output buffer is about to run dry.
}
Version 2, execution is forked and detached from the terminal that launched it:
void go_daemon(){
// Run the program as a daemon.
pid_t pid, sid;
pid = fork(); // Fork off the parent process
if (pid < 0) {
exit(EXIT_FAILURE);
}
if (pid > 0) {
exit(EXIT_SUCCESS); // If child process started ok, exit the parent process
}
umask(0); // Change file mode mask
sid = setsid(); // Create a new session ID for the child process
if (sid < 0) {
// TODO: Log failure
exit(EXIT_FAILURE);
}
if((chdir("/")) < 0){ //Change the working directory to "/"
//TODO: Log failre
exit(EXIT_FAILURE);
}
close(STDIN_FILENO);
close(STDOUT_FILENO);
close(STDERR_FILENO);
}
int main (int argc, char *argv[]) {
go_daemon();
// Subscribe to all system signals for debugging purposes.
for(int i=0; i<64; i++){
signal(i, sighandler);
}
// Sanity checks, error and help messages, etc.
parseParameters(argc, argv);
//Start Signal processing and Audio
initZenGarden();
initAudioIO();
// Keep the program alive.
while(1){
sleep(10);
}
// This is obviously never reached, so far no problems with that...
stopAudioIO();
stopZengarden();
return 0;
}
Trying to run it at startup
I've tried running both versions of the program at startup using a few methods. The outcome is always the same. When the beagle starts up, I can hear sound beign output for a fraction of a second, the sound then stops and the login screen is presented (I have a serial terminal attached to the board and minicom running on my computer). The weirdest thing to me is that the xooky_nabox process is actually kept running after login but there is no sound output...
Here's what I've tried:
Adding a #reboot entry to crontab and launching the program with a trailing ampersand (version 1 of the program):
#reboot xooky_nabox <params> &
Added a start-stop-daemon to crontab (version 1):
#reboot start-stop-daemon -S -b --user daemon -n xooky_nabox -a /usr/bin/xooky_nabox -- <params>
Created a script at /etc/init.d/xooky and did
$chmod +x xooky
$update-rc.d xooky defaults
And tried different versions of the startup script: start-stop-daemon with version 1, calling the program directly with a trailing ampersand (version 1), calling the program directly with no trailing ampersand (version 2).
Also, if I run the program manually from the serial terminal or from a ssh session (usb networking); and then I run top, the program will run fine for a few seconds consuming around 15% cpu. It will then stop outputing sound, and it's cpu consumption will rise to around 30%. My log.txt file shows no signal sent to the program by the OS in this scenario.
When version 2 of the program is ran at startup, the log wil show something like:
Mon Jun 6 02:44:49 2011 Caught signal:18 Continued
Mon Jun 6 02:44:49 2011 Caught signal:15 Terminated
Does anyone have any ideas on how to debug this? Suggestions on how to launch my program at startup?
In version 2,
I think you should open (and dup2) /dev/null to STDIN/STDOUT/STDERR. Just closing the handle would cause problem.
something like this:
int fd = open("/dev/null", O_RDWR);
dup2( fd, STDOUT_FILENO );
(I have no idea what start-stop-daemon do. Can't help version 1, sorry)
There is C function to create a daemon
#include <unistd.h>
int daemon(int nochdir, int noclose);
More information can be found in man pages for daemon(3)
Maybe it will help.
And if you want to launch you daemon when you linux start, you should find out which init version you are using in you distro, but usually, you can just add command to execute you daemon to /etc/init.d/rc (but it seems to be no so good idea). This file is executed by init when linux is starting.
I ended up ditching PortAudio and implementing a JACK client which runs it's own server so this issue was not relevant for me anymore.