I'm running some commands with the C++ system() function:
int system ( const char * command );
How can I collect the standard output from the issued commands?
To be specific, I want to collect the output of the issued command (for example, the directory listing output from issuing the dir command).
Are you looking for returned value (as in "exit status") of the executed command, or for its output (as in "what did it print")?
If the latter, use popen() and pclose() instead.
If the former, look at the return value from system() (and use the documentation for waitpid() to interpret it).
system() returns an int, so just grab it: int rvalue = system(command);
I believe the exact details of what system() will return are system-specific, though.
There are typically two ways for a system program to "return" a value: by writing to stdout, and by returning a status integer at the end of the program. (there are often more ways to return results, eg. by writing to a file or into a database, but I assume those are out of scope here).
For receiving the status code, just check the return value of the system function.
For receiving the output, either redirect it into a file, and read the file afterwards, or use popen.
The return value of system is (ironically) system-dependent, but in POSIX systems (including Linux, etc), it's the same as for wait -- low 8 or 16 bits are the exit status of the child (probably what you mean by "value returned by"), higher bits indicating what kind of signal terminated the child, if any. The URL to the manpage I've given supplies the preprocessor macros you can use to pry apart that return value!
There is no such thing as a "return string" of a program, as you've now clarified in a comment is what you desire; as another answer already mentioned, if you want the text which gets output by the other program, you should use popen instead of system.
Inspired by bmorin's attempt, but working and tested, this snippet will take a char* command and return a char* containing the results of executing that command...
// Calling function must free the returned result.
char* exec(const char* command) {
FILE* fp;
char* line = NULL;
// Following initialization is equivalent to char* result = ""; and just
// initializes result to an empty string, only it works with
// -Werror=write-strings and is so much less clear.
char* result = (char*) calloc(1, 1);
size_t len = 0;
fflush(NULL);
fp = popen(command, "r");
if (fp == NULL) {
printf("Cannot execute command:\n%s\n", command);
return NULL;
}
while(getline(&line, &len, fp) != -1) {
// +1 below to allow room for null terminator.
result = (char*) realloc(result, strlen(result) + strlen(line) + 1);
// +1 below so we copy the final null terminator.
strncpy(result + strlen(result), line, strlen(line) + 1);
free(line);
line = NULL;
}
fflush(fp);
if (pclose(fp) != 0) {
perror("Cannot close stream.\n");
}
return result;
}
I looked into just editing bmorin's code, but would have had to change most lines, so a separate answer seemed more appropriate. Apologies if not. (Amongst other problems, bmorin's code didn't actually accumulate the lines; it printed them to stdout, where I presume they would not be wanted, since system() would have done that; and it returned void in one error path, when the function must return a char*, so the code wouldn't compile. Perhaps most egregious, it freed the result just before returning it.)
system() is declared and defined in libc. You can either read the first link I provided, or do man system at a command prompt in your shell.
I suggest the popen() functions, as said by other people as well,
but this problem is platform specific. the popen() function is
available on operating systems that use the POSIX API. I am not
sure if this command would work on other APIs like WIN32
Here is a code snippet (in plain C) executing a command with popen and returning its output:
char* exec(const char* command) {
FILE* fp;
char* result = NULL;
size_t len = 0;
fflush(NULL);
fp = popen(command, "r");
if (fp == NULL) {
printf("Cannot execute command:\n%s\n", command);
return;
}
while(getline(&result, &len, fp) != -1) {
fputs(result, stdout);
}
free(result);
fflush(fp);
if (pclose(fp) != 0) {
perror("Cannot close stream.\n");
}
return result;
}
Related
The short version: I have a C++ code that uses a C call to fprintf(stdout, some_cpp_str.c_str()) and crashes during it. The first 4 calls are fine, only the 5th crashes, and I have no idea why (suspecting unreadable char inside the string). The 1st code I posted was mostly C, so I posted another one, with only C++ except for the fprintf (code added at the bottom of the question). The crashes occur (consistently) on an embedded device. On my own PC the code runs fine
The long version:
I have a code that reads lines from text, and pushes them into a string vector. TO check my progress, I also fprintf them to the screen after the vector is populated:
int main(){
char err_msg[256], * line = NULL, *in_file = "...", *keyword = "blah";
size_t len = 0;
ssize_t num_bytes_read;
int i = 1;
std::vector<std::string> lines_vector;
FILE * fp = fopen(in_file, "r");
if (!fp) {
fprintf(stdout,"can't open file %s for reading\n", in_file);
goto EXIT;
}
while ((num_bytes_read = getline(&line, &len, fp)) != -1) {
/* if found keyword inside line */
if (strstr(line, keyword)) {
/* add 3 lines (entry heading, entry body, newline)*/
lines_vector.push_back(std::string(line));
for(int lines_to_copy = 2; lines_to_copy > 0; lines_to_copy--) {
if((num_bytes_read = getline(&line, &len, fp)) == -1) {
fprintf(stdout,"can't read line from %s\n", in_file);
goto EXIT;
}
lines_vector.push_back(std::string(line));
}
}
}
fprintf(stdout,"finished reading from file\n");
EXIT:
fclose(fp);
free(line);
for (std::vector<std::string>::iterator it = lines_vector.begin() ; it != lines_vector.end(); ++it, ++i) {
fprintf(stdout, "%d)", i);
fprintf(stdout, "%s", (*it).c_str());
}
return 0;
}
This works fine on my VM, but I also run it on an embedded device, where it always crashes on a specific line. The line is:
certificates local generate name localcert common-name sf country(region) AB auto-regenerate-days 12 auto-regenerate-days-warning 11 e-mail X#Y.com locality(city) Z organization Q organization-unit T scep-password-string 57E6CA35452E72E4D1BC4518260ABFC7 scep-url http://0.0.0.0/X/Y/ state(province) s
I don't think there is a problem in the line itself (as it doesn't crash on my VM). When trying to print it to a file instead of to the screen, it doesn't crash:
for (std::vector<std::string>::iterator it = lines_vector.begin(); it != lines_vector.end(); ++it){
sprintf(tmp, "echo \"%s\" >> /X/Y/Z.txt", (*it).c_str());
OS_run(tmp); // run this command on sh shell
}
Since it crashes only on my embedded and not my VM, I thought the file is somehow corrupted. Could it be that the string has an invalid char inside that crashes fprintf, but not echo?
I tried translating this code into proper C++, but I still get a crash in the middle of the last string. I know mixing C/C++ is not good, but shouldn't c_str() be a proper interface between std::string and char * (which fprintf expects)?
If not this, then what could possibly crash during the fprintf?
int main()
{
std::vector<std::string> lines_vector;
std::ifstream infile(in_file);
std::string line;
int counter = 1;
while (std::getline(infile, line)) {
if (line.find(keyword, 0) != std::string::npos) {
lines_vector.push_back(line);
for(int lines_to_copy = 2; lines_to_copy > 0; lines_to_copy--) {
std::getline(infile, line);
lines_vector.push_back(line);
}
}
}
for (std::vector<std::string>::iterator it = lines_vector.begin(); it != lines_vector.end(); ++it){
fprintf(stdout, "%d)%s", counter++, (*it).c_str());
}
}
On an embedded device, you can expect that dynamic memory allocation fail. That means that you absolutely must control all possible allocations (you should anyway even on non embedded device, but the crash risk is much lower...). You really should have:
while ((num_bytes_read = getline(&line, &len, fp)) != -1) {
...
}
if (line == NULL) {
perror("getline could not allocate buffer");
}
This will not fix anything, but at least you will know what happens.
I have respected your coding style here, making heavy use of the C library and also using goto. But I must advise you not to do that in C++ programs.
C library used to be included in C++ standard library because early C++ implementations were lacking too many functionalities. In modern C++ goto is to be banned, as are all raw C strings and C io functions (except in very special use cases). And C++ come with a version of getline (in header <string>) that directly fills a std::string. You really should try to avoid C construct if learning C++.
Per Ben Voigt's comment, there are correct use case to use old style C library if you want to avoid dynamic allocation. But in that case, you should also avoid std::string and std::vector
Okay, here's my code that seems to fail and doesn't even write at least once, I fail to see what's actually wrong, could anyone point me in the correct direction?
My intention is to perform this write every 10 seconds on an infinite loop
FILE* pFile;
pFile = fopen("Hdd:\\LOGFile.txt", "w");
while(true) {
DWORD TitleID = XamGetCurrentTitleId();
std::ostringstream titleMessageSS;
titleMessageSS << "Here's the current title we're on : " << TitleID << "\r";
std::string titleMessage = titleMessageSS.str(); // get the string from the stream
DWORD dwBytesToWrite = (DWORD)titleMessage.size();
DWORD dwBytesWritten = 0;
fwrite(titleMessage.c_str(), 1 , dwBytesToWrite, pFile);
Sleep(10000);
}
fclose(pFile);
return NULL;
You probably should call fflush(pFile); before your call to Sleep;
but you should better use C++ std::ostream (and use std::flush manipulator).
How do you know that it doesn't write? Is the file empty after the
program stops.
What I suspect is that you're looking at the output file before the
program has stopped. The stream types (both C++ and C) are normally
buffered; fwrite copies the data to a local buffer, and only transfers
it to the OS (system function write under Unix) when the buffer is
full, when you explicitly flush it (see fflush) or when the file is
closed. This is, for example, why we have to check for output errors
after fclose().
I'm doing a system call in c++:
system( "g++ file1.cpp -o test" );
And I'm trying to get the result in the command line into a std::string object. The reason I'm doing that is to analyze the output of g++ in the case where g++ fails because there is an error in the cpp.
Here's the method I've built for that:
std::string CmdLineCall( std::string cmd )
{
FILE* pipe = _popen( cmd.c_str(), "r");
if( pipe == NULL )
{
return "error";
}
char* buffer;
long bufferSize = cmd.size();
size_t code;
std::string result;
buffer = ( char* )malloc( sizeof( char ) * bufferSize );
if( buffer == NULL )
{
return "error";
}
code = fread( buffer, 1, sizeof( buffer ), pipe );
if( code != bufferSize )
{
return "error";
}
result = buffer;
return result;
}
I've debugged it and it goes all the way to the end but result = "".
I've tried with a simple system( "pause" ) command and it works, but it doesn't work.
Can it have something to do with the fact that the g++ call fails because my cpp file is flawed?
Thanks
Using system() is probably not the best choice for this. You might have better results with popen(), since that lets you read the standard output of your command -- but I suspect that won't be enough here either (as I assume the compiler may also print to stderr and popen() doesn't give you that).
Your best bet is to provide your own variant of popen that includes support for both stdout and stderr (and if you search for "popen source" I'm sure you'll have a good starting point).
Edit: on a re-read of your post, I see you are using popen()… so I'm a bit confused about where system() comes into play since you wouldn't use both of them. Regardless, I still feel the problem is that stderr is important and popen() isn't giving that to you, so you'll need to go get it yourself; you could do that by rewriting popen() or you could just redirect your own stderr to the input channel of a pipe before you call popen().
I have a C++ application which dynamically loads plug-in DLLs. The DLL sends text output via std::cout and std::wcout. Qt-based UI must grab all text output from DLLs and display it.
The approach with stream buffer replacement doesn't fully work since DLLs might have different instances of cout/wcout due to run-time libraries differences. Thus I have applied Windows-specific STDOUT redirection as follows:
StreamReader::StreamReader(QObject *parent) :
QThread(parent)
{
// void
}
void StreamReader::cleanUp()
{
// restore stdout
SetStdHandle (STD_OUTPUT_HANDLE, oldStdoutHandle);
CloseHandle(stdoutRead);
CloseHandle(stdoutWrite);
CloseHandle (oldStdoutHandle);
hConHandle = -1;
initDone = false;
}
bool StreamReader::setUp()
{
if (initDone)
{
if (this->isRunning())
return true;
else
cleanUp();
}
do
{
// save stdout
oldStdoutHandle = ::GetStdHandle (STD_OUTPUT_HANDLE);
if (INVALID_HANDLE_VALUE == oldStdoutHandle)
break;
if (0 == ::CreatePipe(&stdoutRead, &stdoutWrite, NULL, 0))
break;
// redirect stdout, stdout now writes into the pipe
if (0 == ::SetStdHandle(STD_OUTPUT_HANDLE, stdoutWrite))
break;
// new stdout handle
HANDLE lStdHandle = ::GetStdHandle(STD_OUTPUT_HANDLE);
if (INVALID_HANDLE_VALUE == lStdHandle)
break;
hConHandle = ::_open_osfhandle((intptr_t)lStdHandle, _O_TEXT);
FILE *fp = ::_fdopen(hConHandle, "w");
if (!fp)
break;
// replace stdout with pipe file handle
*stdout = *fp;
// unbuffered stdout
::setvbuf(stdout, NULL, _IONBF, 0);
hConHandle = ::_open_osfhandle((intptr_t)stdoutRead, _O_TEXT);
if (-1 == hConHandle)
break;
return initDone = true;
} while(false);
cleanUp();
return false;
}
void StreamReader::run()
{
if (!initDone)
{
qCritical("Stream reader is not initialized!");
return;
}
qDebug() << "Stream reader thread is running...";
QString s;
DWORD nofRead = 0;
DWORD nofAvail = 0;
char buf[BUFFER_SIZE+2] = {0};
for(;;)
{
PeekNamedPipe(stdoutRead, buf, BUFFER_SIZE, &nofRead, &nofAvail, NULL);
if (nofRead)
{
if (nofAvail >= BUFFER_SIZE)
{
while (nofRead >= BUFFER_SIZE)
{
memset(buf, 0, BUFFER_SIZE);
if (ReadFile(stdoutRead, buf, BUFFER_SIZE, &nofRead, NULL)
&& nofRead)
{
s.append(buf);
}
}
}
else
{
memset(buf, 0, BUFFER_SIZE);
if (ReadFile(stdoutRead, buf, BUFFER_SIZE, &nofRead, NULL)
&& nofRead)
{
s.append(buf);
}
}
// Since textReady must emit only complete lines,
// watch for LFs
if (s.endsWith('\n')) // may be emmitted
{
emit textReady(s.left(s.size()-2));
s.clear();
}
else // last line is incomplete, hold emitting
{
if (-1 != s.lastIndexOf('\n'))
{
emit textReady(s.left(s.lastIndexOf('\n')-1));
s = s.mid(s.lastIndexOf('\n')+1);
}
}
memset(buf, 0, BUFFER_SIZE);
}
}
// clean up on thread finish
cleanUp();
}
However, this solution appears to have an obstacle - C runtime library, which is locale-dependent. Thus any output sent to wcout isn't reaching my buffer because C runtime truncates strings at non-printable ASCII characters present in UTF-16 encoded strings. Calling setlocale() demonstrates, that C runtime does string re/encoding. setlocale() is no help for me for very reason that there is no knowledge of the language or locale of the text, since plug-in DLLs read from outside the system and there might be different languages mixed.
After giving an N-thought I have decided to drop this solution and revert to cout/wcout buffer replacement and putting requirement for DLLs to call initialization method due to both reasons: UTF16 not passing to my buffer, and then the problem of figuring out encoding in the buffer. However, I am still curious of whether there is a way to get UTF-16 strings through C runtime into pipe 'as is', without locale-dependent conversion?
p.s. any suggestions on cout/wcout redirection to UI rather than the two mentioned approaches are welcome as well :)
Thank you in advance!
The problem here is that the code conversion from wchar_t to char is being done entirely inside the plug-in DLL, by whatever cout/wcout implementation it happens to be using (which as you say may not be the same as the one that the main application is using). So the only way to get it to behave differently is to intercept that mechanism somehow, such as with streambuf replacement.
However, as you imply, any code you write in the main application isn't necessarily going to be compatible with the library implementation that the DLL uses. For example, if you implement a stream buffer in the main application, it won't necessarily be using the same ABI as the stream buffers in the DLL. So this is risky.
I suggest you implement a wrapper DLL that uses the same C++ library version as the plug-in, so it's guaranteed to be compatible, and in this wrapper DLL do the necessary intervention in cout/wcout. It could load the plug-in dynamically, and so could be reusable with any plug-in that uses that library version. Alternatively, you could create some reusable source code that could be compiled specifically for each plug-in, thus producing a sanitized version of each plug-in.
Once the DLL is wrapped, you can substitute a stream buffer into cout/wcout that saves the data to memory, as I think you were originally planning, and not have to mess with file handles at all.
PS: If you ever do need to make a wstream that converts to and from UTF-8, then I recommend using Boost's utf8_codecvt_facet as a very neat way of doing it. It's easy to use, and the documentation has example code.
(In this scenario you would have to compile a version of Boost specifically for the library version that the plug-in is using, but not in the general case.)
I don't know if this is possible, but maybe you could start the DLL in a separate process and capture the output of that process with the Windows equivalent of pipe (whatever that is, but Qt's QProcess should take care of that for you). This would be similar to how Firefox does out of process plugins (default in 3.6.6, but it's been done for a while with 64 bit Firefox and the 32 bit Flash plugin). You'd have to come up with some way to communicate with the DLL in the separate process, like shared memory, but it should be possible. Not necessarily pretty, but possible.
Try:
std::wcout.imbue(std::locale("en_US.UTF-8"));
This is stream-specific, and better than using the global C library setlocale().
However, you may have to tweak the locale name to suit what your runtime supports.
I'm using the following code to try to read the results of a df command in Linux using popen.
#include <iostream> // file and std I/O functions
int main(int argc, char** argv) {
FILE* fp;
char * buffer;
long bufSize;
size_t ret_code;
fp = popen("df", "r");
if(fp == NULL) { // head off errors reading the results
std::cerr << "Could not execute command: df" << std::endl;
exit(1);
}
// get the size of the results
fseek(fp, 0, SEEK_END);
bufSize = ftell(fp);
rewind(fp);
// allocate the memory to contain the results
buffer = (char*)malloc( sizeof(char) * bufSize );
if(buffer == NULL) {
std::cerr << "Memory error." << std::endl;
exit(2);
}
// read the results into the buffer
ret_code = fread(buffer, 1, sizeof(buffer), fp);
if(ret_code != bufSize) {
std::cerr << "Error reading output." << std::endl;
exit(3);
}
// print the results
std::cout << buffer << std::endl;
// clean up
pclose(fp);
free(buffer);
return (EXIT_SUCCESS);
}
This code is giving me a "Memory error" with an exit status of '2', so I can see where it's failing, I just don't understand why.
I put this together from example code that I found on Ubuntu Forums and C++ Reference, so I'm not married to it. If anyone can suggest a better way to read the results of a system() call, I'm open to new ideas.
EDIT to the original: Okay, bufSize is coming up negative, and now I understand why. You can't randomly access a pipe, as I naively tried to do.
I can't be the first person to try to do this. Can someone give (or point me to) an example of how to read the results of a system() call into a variable in C++?
You're making this all too hard. popen(3) returns a regular old FILE * for a standard pipe file, which is to say, newline terminated records. You can read it with very high efficiency by using fgets(3) like so in C:
#include <stdio.h>
char bfr[BUFSIZ] ;
FILE * fp;
// ...
if((fp=popen("/bin/df", "r")) ==NULL) {
// error processing and return
}
// ...
while(fgets(bfr,BUFSIZ,fp) != NULL){
// process a line
}
In C++ it's even easier --
#include <cstdio>
#include <iostream>
#include <string>
FILE * fp ;
if((fp= popen("/bin/df","r")) == NULL) {
// error processing and exit
}
ifstream ins(fileno(fp)); // ifstream ctor using a file descriptor
string s;
while (! ins.eof()){
getline(ins,s);
// do something
}
There's some more error handling there, but that's the idea. The point is that you treat the FILE * from popen just like any FILE *, and read it line by line.
Why would std::malloc() fail?
The obvious reason is "because std::ftell() returned a negative signed number, which was then treated as a huge unsigned number".
According to the documentation, std::ftell() returns -1 on failure. One obvious reason it would fail is that you cannot seek in a pipe or FIFO.
There is no escape; you cannot know the length of the command output without reading it, and you can only read it once. You have to read it in chunks, either growing your buffer as needed or parsing on the fly.
But, of course, you can simply avoid the whole issue by directly using the system call df probably uses to get its information: statvfs().
(A note on terminology: "system call" in Unix and Linux generally refers to calling a kernel function from user-space code. Referring to it as "the results of a system() call" or "the results of a system(3) call" would be clearer, but it would probably be better to just say "capturing the output of a process.")
Anyway, you can read a process's output just like you can read any other file. Specifically:
You can start the process using pipe(), fork(), and exec(). This gives you a file descriptor, then you can use a loop to read() from the file descriptor into a buffer and close() the file descriptor once you're done. This is the lowest level option and gives you the most control.
You can start the process using popen(), as you're doing. This gives you a file stream. In a loop, you can read using from the stream into a temporary variable or buffer using fread(), fgets(), or fgetc(), as Zarawesome's answer demonstrates, then process that buffer or append it to a C++ string.
You can start the process using popen(), then use the nonstandard __gnu_cxx::stdio_filebuf to wrap that, then create an std::istream from the stdio_filebuf and treat it like any other C++ stream. This is the most C++-like approach. Here's part 1 and part 2 of an example of this approach.
I'm not sure you can fseek/ftell pipe streams like this.
Have you checked the value of bufSize ? One reason malloc be failing is for insanely sized buffers.
Thanks to everyone who took the time to answer. A co-worker pointed me to the ostringstream class. Here's some example code that does essentially what I was attempting to do in the original question.
#include <iostream> // cout
#include <sstream> // ostringstream
int main(int argc, char** argv) {
FILE* stream = popen( "df", "r" );
std::ostringstream output;
while( !feof( stream ) && !ferror( stream ))
{
char buf[128];
int bytesRead = fread( buf, 1, 128, stream );
output.write( buf, bytesRead );
}
std::string result = output.str();
std::cout << "<RESULT>" << std::endl << result << "</RESULT>" << std::endl;
return (0);
}
To answer the question in the update:
char buffer[1024];
char * line = NULL;
while ((line = fgets(buffer, sizeof buffer, fp)) != NULL) {
// parse one line of df's output here.
}
Would this be enough?
First thing to check is the value of bufSize - if that happens to be <= 0, chances are that malloc returns a NULL as you're trying to allocate a buffer of size 0 at that point.
Another workaround would be to ask malloc to provide you with a buffer of the size (bufSize + n) with n >= 1, which should work around this particular problem.
That aside, the code you posted is pure C, not C++, so including is overdoing it a little.
check your bufSize. ftell can return -1 on error, and this can lead to nonallocation by malloc with buffer having a NULL value.
The reason for the ftell to fail is, because of the popen. You cant search pipes.
Pipes are not random access. They're sequential, which means that once you read a byte, the pipe is not going to send it to you again. Which means, obviously, you can't rewind it.
If you just want to output the data back to the user, you can just do something like:
// your file opening code
while (!feof(fp))
{
char c = getc(fp);
std::cout << c;
}
This will pull bytes out of the df pipe, one by one, and pump them straight into the output.
Now if you want to access the df output as a whole, you can either pipe it into a file and read that file, or concatenate the output into a construct such as a C++ String.