I'm developing a cross platform project currently. On windows i had a class that ran a process/script (using a commandline), waited for it to end, and read everything from it's stdout/stderr to a buffer. I then printed the output to a custom 'console'. Note: This was not a redirection of child stdout to parent stdout, just a pipe from child stdout to parent.
I'm new to OSX/unix-like api's but i can understand the canonical way of doing something like this is forking and piping stdouts together. However, i dont want to redirect it to stdout and i would like to capture the output.. It should work pretty much like this (pseudocode, resemblance with unix functions purely coincidental):
class program
{
string name, cmdline;
string output;
program(char * name, char * cmdline)
: name(name), cmdline(cmdline) {};
int run()
{
// run program - spawn it as a new process
int pid = exec(name, cmdline);
// wait for it to finish
wait(pid);
char buf[size];
int n;
// read output of program's stdout
// keep appending data until there's nothing left to read
while (read(pid, buf, size, &n))
output.append(buf, n);
// return exit code of process
return getexitcode(pid);
}
const string & getOutput() { return output; }
};
How would i go about doing this on OSX?
E:
Okay so i studied the relevant api's and it seems that some kind of fork/exec combo is unavoidable. Problem at hand is that my process is very large and forking it really seems like a bad idea (i see that some unix implementations can't do it if the parent process takes up 50%+ of the system ram).
Can't i avoid this scheme in any way? I see that vfork() might be a possible contender, so maybe i could try to mimic the popen() function using vfork. But then again, most man pages state that vfork might very well just be fork()
You have a library call to do just that: popen. It will provide you with a return value of a file descriptor, and you can read that descriptor till eof. It's part of stdio, so you can do that on OSX, but other systems as well. Just remember to pclose() the descriptor.
#include <stdio.h>
FILE * popen(const char *command, const char *mode);
int pclose(FILE *stream);
if you want to keep output with absolutely no redirection, the only thing we can think of is using something like "tee" - a command which splits the output to a file but maintains its own stdout. It's fairly easy to implement that in code as well, but it might not be necessary in this case.
Related
If you want to spawn a Windows console in an otherwise SUBSYSTEM:WINDOWS application you can use this code:
if (AllocConsole())
{
FILE* file = nullptr;
_wfreopen_s(&file, L"CONIN$", L"r", stdin);
_wfreopen_s(&file, L"CONOUT$", L"w", stdout);
_wfreopen_s(&file, L"CONOUT$", L"w", stderr);
}
The _wfreopen_s function maps stdin to CONIN$ and provides a pointer to pointer in the file variable (which we are effectively discarding).
What I'd like to do is instead map an input from something other than stdin, for example, another file stream and then write that stream to CONIN$.
For a larger picture of what I'm trying to do here, I've got a secondary thread running std::getline(std::cin... which blocks. I'd like the thread context object to just send a \n to the console to break the blocking call.
If there are other ideas, I'm open. The alternative currently is that I print a message to the console that says "Shutting down, press ENTER to quit..." Which, I guess, also works ;)
What I tried was using the FILE* conin = new FILE(); and then did a memcpy to fill it with a \n and then I used WriteFile to that pointer, thinking that it might write the file stream out to CONIN$, and while the code compiles, and the contents of the FILE* appears to be correct (0x0a), it does not appear to send that stream to the console.
I tested this by having std::cout above and below the code testing the stream write. If it works, I'd expect the two lines to be on separate lines, but they always show up on the same, suggesting that I'm not sending the file stream.
Thanks for reading!
You should not discard the FILE* handle, otherwise you won't be able to manipulate it, in particular you won't be able to properly flush/close it if required.
If you're working with threads, simply give the FILE* to the thread that requires it. Threads share the same memory space.
If you're working with processes, then you should create a pipe between the two processes involved (see Win32 API for CreatePipe for details), and connect one's stdout to the other's stdin.
I am writing a console application that accepts input (one-line commands) from stdin. This application reads input in a dedicated thread, all input is stored in a queue and later processed by the main thread in a safe way. When the exit command is entered by the user, it is intercepted by the input thread which stops listening for new input, the thread is joined into the main one, and the application stops as requested.
Now I am containerizing this application, but I still want to be able to attach to the container and input commands from stdin, so I specified tty and stdin_open to be true in my docker compose service file, and that did the trick.
But I also want docker compose to be able to gracefully stop the application, so I decided to implement sigTerm() in my application so that it can receive the signal from docker compose and gracefully stop, however I'm stuck on that part, because the input thread is blocking while waiting for input on stdin. I can properly receive the signal, that's not at all the point here, but I'm looking for a way to be able to properly stop my containerized application while still being able to input commands from the keyboard.
My application could be simplified like that :
void gracefulStop() {
while (getThreadCount() > 1) { // this function exists somewhere else.
if (userInputThread.joinable()) {
userInputThread.join();
removeFromThreadCount(); // this function exists somewhere else.
}
std::this_thread::sleep_for(std::chrono::seconds(1));
}
exit(SUCCESS);
}
void sigTerm(int s) {
// Maybe do some stuff here, but what...
gracefulStop();
}
void userInputLoopThreadFunc() {
addToThreadCount(); // this function exists somewhere else.
while (keepGoing) {
char buf[4096];
if (!fgets(buf, sizeof(buf), stdin)) {
break; // we couldn't read from stdin, stop trying.
}
std::string input = std::string(buf); // we received a command
// Intercept exit command
if (input.starts_with("exit")) {
keepGoing = false;
}
// IRL there's thread safety
userInputQueue.push(input); // this will be processed by mainLoop() later
}
}
int main(int argc, char **argv) {
// Register the signal
signal(SIGTERM, sigTerm);
// Begin listening to user input
userInputThread = std::thread(&userInputLoopThreadFunc, this);
// this mainLoop function contains the core of the application
// as well as the processing code of the user input
mainLoop();
// if mainLoop function returned, we received the 'exit' command
gracefulStop();
}
I've read multiple question/answers like this one about non-blocking user input (the accepted answer advises to use a dedicated thread for input, which is what I am doing), or this other one about how to stop reading stdin, and the accepted answer seems promising but :
using ncurses for what I'm trying to do seems really overkill
If using select() and the timeout mechanism described, what would happen if the timeout occurs while typing a command?
Also I've read about the c++20 jthread here :
The class jthread represents a single thread of execution. It has the same general behavior as std::thread, except that jthread automatically rejoins on destruction, and can be cancelled/stopped in certain situations.
But I'm not sure that would help me here.
I'm thinking about multiple possibilities to solve my issue :
Find a way to send a newline character to the stdin of my application without user interaction, would be hackish if at all possible but would probably unblock fgets.
Kill the thread, I understand killing a thread is considered a bad practice, but since the only thing I'm doing here is stopping the application, maybe I can live with that, would there be any side effect? How would I do that?
Rewriting user input in another way (unknown to me yet, jthread, something else?) that would allow sigTerm() to stop the application.
Maybe use ncurses (would that really help me to stop the application by receiving a signal?)
Go with select() and the timeout mechanism and live with the risk of an interrupted input
Give up on user input and have some vacation time.
You can close stdin in your signal handler. fgets will then return immediately (and presumably, return NULL).
The good news is that close is on the list of functions that are safe to call from a signal handler (it's a pretty restrictive list). Happy days.
There's an alternative based around EINTR, but it looks messy since you don't know for certain that fgets will actually return when it gets it.
Also, closing stdin should still work should you switch to using cin and getline, which would definitely improve your code (*). That probably returns and sets badbit when you close stdin, although the code can be made more robust than by checking for that alone. Perhaps just set a (volatile) flag in your signal handler and test that.
(*) Because getline can read into a std::string, which means it can read arbitrary long lines without worrying about allocating a fixed-size buffer that is 'big enough'.
Consider we have some_function and it prints result to stdout instead returning it.Changing it's defination is out of our scope and there's no alternative to it. We're left with option of reading it from stdout. So the question.
How to read stdout of C++ program in itself.
It is possible to get pid I searched if we can get fd of the same programm but I'm not able to find anything.
#include <unistd.h>
#include <sys/types.h>
#include <iostream>
void some_function(){
std::cout<<"Hello World";
}
int main(){
int pid = ::getpid();
string s = //What to write here.
cout<<"Printing";
some_function(); //This function prints "Hello World" to screen
cout<<s; //"PrintingHello World"
return 0;
}
How to attach pipe to same process i.e instead of creating child process.
Some might think of creating child process and call some_function in it, to be able to read its stdout in parent process, but No, some_function depends on process which calls it and hence we want to call it the very process instead of creating child process.
This isn't hard to do, but IMO it's quite a hack, and it won't work with a multithreaded program:
// make a temp file to store the function's stdout
int newStdOut = mkstemp( "/tmp/stdout.XXXXXXX" );
// save the original stdout
int tmpStdOut = dup( STDOUT_FILENO );
// clear stdout
fflush( stdout );
// now point the stdout file descriptor to the file
dup2( newStdOut, STDOUT_FILENO );
// call the function we want to collect the stdout from
some_function();
// make sure stdout is empty
fflush( stdout );
// restore original stdout
dup2( tmpStdOut, STDOUT_FILENO );
// the tmp file now contains whatever some_function() wrote to stdout
Error checking, proper headers, syncing C stdout with C++ cout, and reading from and cleaning up the temp file are left as exercises... ;-)
Note that you can't safely use a pipe - the function can write enough to fill up the pipe, and you can't read from the pipe because you've called the function.
How to read stdout of C++ program in itself?
There are very few reasons to do that and that is usually (but not always) a design bug.
Be aware of an important thing (at least in a single-threaded program). If your program is both reading from its "stdout" and writing (as usual) in it, it could be stuck in a deadlock: unable to read so not reaching any output routine, (or unable to write because the pipe is full).
So a program which both reads and writes the same thing (actually, the two sides of the same pipe(7)) should use some multiplexing call like poll(2). See also this.
Once you understand that, you'll have some event loop. And before that, you'll make a pipe(7) using pipe(2) (and dup2(2)).
However, pipe to self is a good thing in some signal(7) handling (see signal-safety(7)). That trick is even recommended in Qt Unix signal handling.
Read more about Unix system programming, e.g. ALP or some newer book. Read also intro(2) & syscalls(2).
I have looked for pipe and it requires fd
Wrong. Read much more carefully pipe(2); on success it fills an array of two file descriptors. Of course it could fail (see errno(3) & perror(3) & strerror(3))
Maybe you just need popen(3). Or std::ostringstream. Or open_memstream(3).
Consider we have some_function and it prints result to stdout instead returning it. Changing it's definition is out of our scope and there's no alternative to it
If some_function is your code, or is some free software, you could and probably should improve it to give a result somewhere....
I am writing a baby program for practice. What I am trying to accomplish is basically a simple little GUI which displays services (for Linux); with buttons to start, stop, enable, and disable services (Much like the msconfig application "Services" tab in Windows). I am using C++ with Qt Creator on Fedora 21.
I want to create the GUI with C++, and populating the GUI with the list of services by calling bash scripts, and calling bash scripts on button clicks to do the appropriate action (enable, disable, etc.)
But when the C++ GUI calls the bash script (using system("path/to/script.sh")) the return value is only for exit success. How do I receive the output of the script itself, so that I can in turn use it to display on the GUI?
For conceptual example: if I were trying to display the output of (systemctl --type service | cut -d " " -f 1) into a GUI I have created in C++, how would I go about doing that? Is this even the correct way to do what I am trying to accomplish? If not,
What is the right way? and
Is there still a way to do it using my current method?
I have looked for a solution to this problem but I can't find information on how to return values from Bash to C++, only how to call Bash scripts from C++.
We're going to take advantage of the popen function, here.
std::string exec(char* cmd) {
FILE* pipe = popen(cmd, "r");
if (!pipe) return "ERROR";
char buffer[128];
std::string result = "";
while(!feof(pipe)) {
if(fgets(buffer, 128, pipe) != NULL)
result += buffer;
}
pclose(pipe);
return result;
}
This function takes a command as an argument, and returns the output as a string.
NOTE: this will not capture stderr! A quick and easy workaround is to redirect stderr to stdout, with 2>&1 at the end of your command.
Here is documentation on popen. Happy coding :)
You have to run the commands using popen instead of system and then loop through the returned file pointer.
Here is a simple example for the command ls -l
#include <stdio.h>
#include <stdlib.h>
int main() {
FILE *process;
char buff[1024];
process = popen("ls -l", "r");
if (process != NULL) {
while (!feof(process)) {
fgets(buff, sizeof(buff), process);
printf("%s", buff);
}
pclose(process);
}
return 0;
}
The long approach - which gives you complete control of stdin, stdout, and stderr of the child process, at the cost of fairly significant complexity - involves using fork and execve directly.
Before forking, set up your endpoints for communication - pipe works well, or socketpair. I'll assume you've invoked something like below:
int childStdin[2], childStdout[2], childStderr[2];
pipe(childStdin);
pipe(childStdout);
pipe(childStderr);
After fork, in child process before execve:
dup2(childStdin[0], 0); // childStdin read end to fd 0 (stdin)
dup2(childStdout[1], 1); // childStdout write end to fd 1 (stdout)
dup2(childStderr[1], 2); // childStderr write end to fd 2 (stderr)
.. then close all of childStdin, childStdout, and childStderr.
After fork, in parent process:
close(childStdin[0]); // parent cannot read from stdin
close(childStdout[1]); // parent cannot write to stdout/stderr
close(childStderr[1]);
Now, your parent process has complete control of the std i/o of the child process - and must safely multiplex childStdin[1], childStdout[0], and childStderr[0], while also monitoring for SIGCLD and eventually using a wait-series call to check the process termination code. pselect is particularly good for dealing with SIGCLD while dealing with std i/o asynchronously. See also select or poll of course.
If you want to merge the child's stdout and stderr, just dup2(childStdout[1], 2) and get rid of childStderr entirely.
The man pages should fill in the blanks from here. So that's the hard way, should you need it.
My native C++ Win32 program spawns a worker process and needs to pass a huge configuration string to it. Currently it just passes the string as a command line to CreateProcess(). The problem is the string is getting longer and now it doesn't fit into the 32K characters limitation imposed by Windows.
Of course I could do something like complicating the worker process start - I use the RPC server in it anyway and I could introduce an RPC request for passing the configuration string, but this will require a lot of changes and make the solution not so reliable. Saving the data into a file for passing is also not very elegant - the file could be left on the filesystem and become garbage.
What other simple ways are there for passing long strings to a worker process started by my program on Windows?
One possible strategy is to create a named Pipe and pass the handle ( or pipe name) to the other process. Then use normal Read\Write operations on Pipe to extract the data.
There are several good answers already, but the easiest way is to save it in a file, and pass the filename in the command line.
As well as being simple, an advantage of this approach is that the apps will be very loosely coupled (you'll potentially be able to use the child application stand-alone in other ways, rather than always having to launch it from a program that knows how to pipe data into it via a specialised interface)
If you want to be sure that the file is cleaned up after processing, mark it for deletion on the next reboot. THen if anybody forgets to clean it up, the OS will deal with it for you on the next reboot.
I would prefer Boost's message queue. It's extremely simple yet sophisticated. Here's example:
#include <boost/interprocess/ipc/message_queue.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <boost/shared_ptr.hpp>
using namespace boost::interprocess;
// ------------------------------------------------------------------------------
// Your worker:
// ------------------------------------------------------------------------------
try {
message_queue::remove("NAME_OF_YOUR_QUEUE");
boost::shared_ptr<message_queue> mq(new message_queue(create_only, "NAME_OF_YOUR_QUEUE", 65535, 32));
char message[1024];
std::size_t size_received;
unsigned int priority;
if (mq->timed_receive(&message, sizeof(message), size_received, priority, boost::posix_time::ptime(boost::posix_time::second_clock::universal_time()) + boost::posix_time::seconds(1))) {
std::string s(message); // s now contains the message.
}
} catch (std::exception &) {
// ...
}
// ------------------------------------------------------------------------------
// And the sender:
// ------------------------------------------------------------------------------
try {
boost::shared_ptr<message_queue> mq(new message_queue(create_only, "NAME_OF_YOUR_QUEUE", 1024, 1024));
std::stringstream message;
message << "the very very very long message you wish to send over";
while (!mq.try_send(message.str().c_str(), message.str().length(), 0))
::Sleep(33);
} catch (std::exception &) {
// ...
}
Use shared memory. Pass to a worker process name of shared memory object. Another solution is to use WM_COPYDATA message.
How about reading it from stdin :) It seems to work for the Unix folks.
Guaranteed a lot easier than passing pipe names/handles around!
Here is some official code from MSDN for creating child processes with I/O pipes.
Is it a possibility to set up a named shared memory segment?
http://msdn.microsoft.com/en-us/library/aa366551(VS.85).aspx
You could use an inheritable handle to a section object. In your parent process create a section object (CreateFileMapping) and specify that its handle is to be inherited by the child process; then pass the handle value to the child process on the command line. The child process can then open the section object (OpenFileMapping). Though I would prefer a named section object as the semantics of using it are easier to understand.