I am working on a project that has multiple C++ executables that communicate using named pipes. The main application (App1) spawns the rest of the applications. When spawning, it closes STDIN for the children using:
close(STDIN_FILENO);
And it redirects STDOUT and STDERR to other files that are specific to the child processes. This makes it so that the output from App1 is only from App1 and none of the children. It also allows App1 to accept input from STDIN and not let it get captured by the child processes.
One of the child processes is a Qt application. When spawned, it is using as much CPU as it can, slowing my computer considerably. If I do not close STDIN for the child processes, this behavior stops (but the children capture STDIN instead of the main process, which I don't want).
Why does this happen and how can I prevent the Qt applications from using all the CPU cycles?
Maybe give the Qt app what it wants? Use dup2 after fork but before exec? dup2 will replace a given file descriptor with another so you can replace stdin with a file. Quick example:
if(fork() == 0)
{
int somefd = open("somefile", O_RDONLY);
// replace stdin (0) with somefd before exec-ing
if(dup2(somefd, 0) == -1)
{
// cunning plan failed
}
// exec Qt app here
}
I think I figured out what the issue was while fixing another issue I was having. I was closing the STDIN file descriptor before redirecting the STDERR and STDOUT file descriptors. This was messing up the indexes that are used when I used freopen() to redirect them.
I moved the close() of STDIN to after the redirection, and don't seem to have the problem anymore.
Related
In my c++ windows app I start multiple child processes and I want them to inherit parent's stdout/stderr, so that if output of my app is redirected to some file then that file would also contain output of all child processes that my app creates.
Currently I do that using CreateProcess without output redirection. MSDN has a sample how to redirect output: Creating a Child Process with Redirected Input and Output, but I want to see what alternative do I have. Simplest is to use system and call it from a blocking thread that waits for child to exit. All output is then piped back to parent's stdout/stderr, however in parent process I do not have a chance to process stdout data that comes from child.
There are also other functions to start processes on windows: spawn, exec, which might be easier to port to posix systems.
What should I use if I want it to work on linux/osx? What options do I have if I want it to work on UWP aka WinRT? I might be totally ok with system called from a blocking thread, but perhaps I'd prefer to be able to have more control on process PID (to be able to terminate it) and process stdout/stderr, to prepend each line with child##: for example.
The boost libraries recently released version 1.64 which includes a new boost::process library.
In it, you're given a C++ way to be able to redirect output to a pipe or asio::streambuf, from which you can create a std::string or std::istream to read whatever your child process wrote.
You can read up on boost::process tutorials here, which shows some simple examples of reading child output. It does make heavy use of boost::asio, so I highly recommend you read up on that too.
I am writing a program in openFrameworks a c++ framework. I want to start another app and communicate with it over stdin and stdout. I can start a new thread conveniently using the ofThread class. I had planned on creating two pipes and redirecting the std in and out of the thread to the pipes (using dup2), but unfortunately, this redirects the pipes for the whole app, not just the thread.
Is there a way I can start another app and be able to reads its output and provide it input?
Instead of another thread you'll need to create a child process using the fork() function (which might involve another thread intrinsically).
The difference is, that fork creates a complete copy of the parent process environment that should be shown on an exec() call within scope of the child process, while just exec() from a thread tries to share all the resource from it's parent process (thread) and thus might lead to unexpected concurrency (race conditon) problems.
If your "another app" is implemented as a subthread within your existing program, you don't need to redirect stdin and stdout to communicate with it over pipes. Just pass the pipe file descriptors to the subthread when you start it up. (You can use fdopen to wrap file descriptors in FILE objects. If you have dup2 and pipe, you have fdopen as well.)
My goal is to:
Pipe stdin to stdin of child process.
Pipe stdout of child process to stdout.
Pipe stderr of chile process to stderr.
I have looked at these:
http://www.jukie.net/bart/blog/popenRWE
and
http://jineshkj.wordpress.com/2006/12/22/how-to-capture-stdin-stdout-and-stderr-of-child-program/
but am having trouble doing what I listed.
If you want to connect the child process's stdin/stdout/stderr to your stdin/stdout/stderr you don't have to do anything, it inherits them automatically.
Note that this doesn't give your application any access to the data -- it just goes directly between the child process application and the original streams. So it's not really "wrapping" anything.
I'm working on an audio encoder cgi script that utilises libmp3lame.
I'm writing in a mixture of C/C++.
I plan to have an entry-point cgi that can spawn multiple encoding processes that run in the background. I need the encoding processes to be asynchronous as encoding can take several hours but I need the entry-point cgi to return instantly so the browser can continue about its business.
I have found several solutions for this (some complete/ some not) but there are still a few things I'd like to clear up.
Solution 1 (easiest): The entry-point cgi is a bash script which can then run a C++ process cgi in the background by sending the output to /dev/null/ 2/&>1& (simples! but not very elegant).
Solution 2: Much like solution 1, except the entry-point cgi is in C++ and uses system() to run the proc/s and send the output to /dev/null/ 2/&>1& again.
[question] This works well but I'm not sure if shared hosting companies allow use of the system() function. Is this the case?
Solution 3 (incomplete): I've looked into using fork()/pthread_create() to spawn separate threads which seems more elegant as I can stay in the realms of C. The only problem being: It seems that the parent thread doesn't exit until all child threads have returned.
[question] Is there any way to get the parent thread to exit whilst allowing child threads to continue in the background.
[idea] Maybe I can send the child proc/s output to the black hole! Can I simply redirect stdout to /dev/null. If so, how do I do this?
I hope this makes sense to someone. I'm still a bit of a noob with C stuff so I may be missing very basic concepts (please have mercy!).
I'd be very grateful of any advise on this matter.
Many thanks in advance,
Josh
You probably want the standard Unix daemon technique, involving a double fork:
void daemonize(void)
{
if (fork()) exit(0); // fork. parent exits.
setsid(); // become process group leader
if (fork()) _exit(0); // second parent exits.
chdir("/"); // just so we don't mysteriously prevent fs unmounts later
close(0); // close stdin, stdout, stderr.
close(1);
close(2);
}
Looks like modern Linux machines have a daemon() library function that presumably does the same thing.
It's possible that the first exit should be _exit, but this code has always worked for me.
I am wrapping existing C++ code from a BSD project in our own custom wrapper and I want to integrate it to our code with as few changes as possible. This code uses fprintf to print to stderr in order to log / report errors.
I want to redirect this to an alternative place within the same process. On Unix I have done this with a socketpair and a thread: one end of the socket is where I send stderr (via a call to dup2) and the other end is monitored in a thread, where I can then process the output.
This does not work on Windows though because a socket is not the same as a file handle.
All documents I have found on the web show how to redirect output from a child process, which is not what I want. How can I redirect stderr within the same process getting a callback of some sort when output is written? (and before you say so, I've tried SetStdHandle but cannot find any way to make this work)...
You can use a similar technique on Windows, you just need to use different words for the same concepts. :) This article: http://msdn.microsoft.com/en-us/library/ms682499.aspx uses a win32 pipe to handle I/O from another process, you just have to do the same thing with threads within the same process. Of course, in your case all output to stderr from anywhere in the process will be redirected to your consumer.
Actually, other pieces of the puzzle you may need are _fdopen and _open_osfhandle. In fact, here's a related example from some code I released years ago:
DWORD CALLBACK DoDebugThread(void *)
{
AllocConsole();
SetConsoleTitle("Copilot Debugger");
// The following is a really disgusting hack to make stdin and stdout attach
// to the newly created console using the MSVC++ libraries. I hope other
// operating systems don't need this kind of kludge.. :)
stdout->_file = _open_osfhandle((long)GetStdHandle(STD_OUTPUT_HANDLE), _O_TEXT);
stdin->_file = _open_osfhandle((long)GetStdHandle(STD_INPUT_HANDLE), _O_TEXT);
debug();
stdout->_file = -1;
stdin->_file = -1;
FreeConsole();
CPU_run();
return 0;
}
In this case, the main process was a GUI process which doesn't start with stdio handles at all. It opens a console, then shoves the right handles into stdout and stdin so the debug() function (which was designed as a stdio interactive function) can interact with the newly created console. You should be able to open some pipes and do the same sort of thing to redirect stderr.
You have to remember that what MSVCRT calls "OS handles" are not Win32 handles, but another layer of handles added just to confuse you. MSVCRT tries to emulate the Unix handle numbers where stdin = 0, stdout = 1, stderr = 2 and so on. Win32 handles are numbered differently and their values always happen to be a multiple of 4. Opening the pipe and getting all the handles configured properly will require getting your hands messy. Using the MSVCRT source code and a debugger is probably a requirement.
You mention that you don't want to use a named pipe for internal use; it's probably worth poining out that the documentation for CreatePipe() states, "Anonymous pipes are implemented using a named pipe with a unique name. Therefore, you can often pass a handle to an anonymous pipe to a function that requires a handle to a named pipe." So, I suggest that you just write a function that creates a similar pipe with the correct settings for async reading. I tend to use a GUID as a string (generated using CoCreateGUID() and StringFromIID()) to give me a unique name and then create the server and client ends of the named pipe with the correct settings for overlapped I/O (more details on this, and code, here: http://www.lenholgate.com/blog/2008/02/process-management-using-jobs-on-windows.html).
Once I have that I wire up some code that I have to read a file using overlapped I/O with an I/O Completion Port and, well, then I just get async notifications of the data as it arrives... However, I've got a fair amount of well tested library code in there that makes it all happen...
It's probably possible to set up the named pipe and then just do an overlapped read with an event in your OVERLAPPED structure and check the event to see if data was available... I don't have any code available that does that though.