Today I have been facing a critical bug in my program, it is just a tiny code that uses popen() to pass the commands and pipe its results to a file descriptor where I use fgets() to read the result, The problem for date command my program is been hanging(waiting) and the reason is as we know after issuing date it will show the current date and issues a yes or no question like the below.
The current date is: Fri 08/21/2015
Enter the new date: (mm-dd-yy)----this was shitting my code!!!!
I just want my program to skip this questionnaire.
Note: the string "command" in the below code is what I will receive from the user.
FILE *in;
char buff[512];
string cmd;
command += " 2>&1";
if (!(in = popen(command.c_str(), "r"))) {
status = "0"; // my logic don't bother:)
}
else {
while (fgets(buff, sizeof(buff), in) != NULL) {
cmd += buff;
}
}
Thanks to Petesh.
I have changed to 2>&1<nul instead of 2>&1, the program isn't hanging now,
The key to this solution is adding <nul gives some default value as an answer to the prompts.
Related
I have a simple C++ application which is supposed to read lines from a POSIX named pipe:
#include<iostream>
#include<string>
#include<fstream>
int main() {
std::ifstream pipe;
pipe.open("in");
std::string line;
while (true) {
std::getline(pipe, line);
if (pipe.eof()) {
break;
}
std::cout << line << std::endl;
}
}
Steps:
I create a named pipe: mkfifo in.
I compile & run the C++ code using g++ -std=c++11 test.cpp && ./a.out.
I feed data to the in pipe:
sleep infinity > in & # keep pipe open, avoid EOF
echo hey > in
echo cats > in
echo foo > in
kill %1 # this closes the pipe, C++ app stops on EOF
When doing this under Linux, the application successfully displays output after each echo command as expected (g++ 8.2.1).
When trying this whole process on macOS, output is only displayed after closing the pipe (i.e. after kill %1).
I started suspecting some sort of buffering issue, so i've tried disabling it like so:
std::ifstream pipe;
pipe.rdbuf()->pubsetbuf(0, 0);
pipe.open("out");
With this change, the application outputs nothing after the first echo, then prints out the first message after the second echo ("hey"), and keeps doing so, alwasy lagging a message behind and displaying the message of the previous echo instead of the one executed.
The last message is only displayed after closing the pipe.
I found out that on macOS g++ is basically clang++, as
g++ --version yields: "Apple LLVM version 10.0.1 (clang-1001.0.46.3)".
After installing the real g++ using Homebrew, the example program works, just like it did on Linux.
I am building a simple IPC library built on named pipes for various reasons, so this working correctly is pretty much a requirement for me at this point.
What is causing this weird behaviour when using LLVM? (update: this is caused by libc++)
Is this a bug?
Is the way this works on g++ guaranteed by the C++ standard in some way?
How could I make this code snippet work properly using clang++?
Update:
This seems to be caused by the libc++ implementation of getline().
Related links:
Why does libc++ getline block when reading from pipe, but libstdc++ getline does not?
https://bugs.llvm.org/show_bug.cgi?id=23078
The questions still stand though.
I have worked around this issue by wrapping POSIX getline() in a simple C API and simply calling that from C++.
The code is something like this:
typedef struct pipe_reader {
FILE* stream;
char* line_buf;
size_t buf_size;
} pipe_reader;
pipe_reader new_reader(const char* pipe_path) {
pipe_reader preader;
preader.stream = fopen(pipe_path, "r");
preader.line_buf = NULL;
preader.buf_size = 0;
return preader;
}
bool check_reader(const pipe_reader* preader) {
if (!preader || preader->stream == NULL) {
return false;
}
return true;
}
const char* recv_msg(pipe_reader* preader) {
if (!check_reader(preader)) {
return NULL;
}
ssize_t read = getline(&preader->line_buf, &preader->buf_size, preader->stream);
if (read > 0) {
preader->line_buf[read - 1] = '\0';
return preader->line_buf;
}
return NULL;
}
void close_reader(pipe_reader* preader) {
if (!check_reader(preader)) {
return;
}
fclose(preader->stream);
preader->stream = NULL;
if (preader->line_buf) {
free(preader->line_buf);
preader->line_buf = NULL;
}
}
This works well against libc++ or libstdc++.
As discussed separately, a boost::asio solution would be best, but your question is specifically about how getline is blocking, so I will talk to that.
The problem here is that std::ifstream is not really made for a FIFO file type. In the case of getline(), it is trying to do a buffered read, so (in the initial case) it decides the buffer does not have enough data to reach the delimiter ('\n'), calls underflow() on the underlying streambuf, and that does a simple read for a buffer-length amount of data. This works great for files because the file's length at a point in time is a knowable length, so it can return EOF if there's not enough data to fill the buffer, and if there is enough data, it simply returns with the filled buffer. With a FIFO, however, running out of data does not necessarily mean EOF, so it doesn't return until the process that writes to it closes (this is your infinite sleep command that holds it open).
A more typical way to do this is for the writer to open and close the file as it reads and writes. This is obviously a waste of effort when something more functional like poll()/epoll() is available, but I'm answering the question you're asking.
I am trying to display the results of my commands live. I am already able to retrieve the results, but I want a live stream. Whenever I put a while loop into my code to repeat the command, a window opens, the command executes, then the window closes, then another window opens and the process repeats until the program crashes. I want the command prompt to run in the background while receiving commands because I am running a QT gui in front of it.
Here is my code currently:
cmd = _popen("snmpget -v 2c -c public 192.168.127.101 .1.3.6.1.4.1.8691.8.4.6.1.1.3.1.1.4.1", "r"); //input command in format (snmp_function -v version -c community_string IP_address OID)
if (cmd == NULL)
{
perror("_popen");
exit(EXIT_FAILURE);
}
while (fgets(result, sizeof(result), cmd)) //gets full responce string to snmpget and passes string to result variable
{
std::string str = result; //sets string to values in variable result
std::regex rgx("\"([^\"]*)\""); //retrieves only the section of the output string that is between double air quotes
std::smatch match;
std::string buffer;
std::stringstream ss(str);
std::vector<std::string> strings;
//necessary? whitespace
while (ss >> buffer)
strings.push_back(buffer);
for (auto& i : strings)
{
if (std::regex_match(i, match, rgx))
{
std::ssub_match submatch = match[1];
std::string str = submatch.str();
for (size_t j=0; j<str.length(); j++)
{
output[j] = str[j];
}
}
}
}
_pclose(cmd);
The std::string stuff is just to get the part of the return value I want.
I can normally hold the window open by putting in a getchar() at the end of a while loop that contains this, but I don't want to be reliant on the user pressing enter to get an updated result.
I suggest that you use a nested cmd when you execute the first command. Something like:
cmd.exe /c cmd.exe /k "Put the command you want to execute here!"
Im not sure if quoting the command is necessary but I know that it does work with quotes so that would mean you change your first line to:
cmd = _popen("cmd.exe /c cmd.exe /k \"snmpget -v 2c -c public 192.168.127.101 .1.3.6.1.4.1.8691.8.4.6.1.1.3.1.1.4.1\"", "r"); //Note the escaped quotes
The above is untested but I believe it should give you the intended result. Note that this opens an inner cmd each time so with this solution you might want to keep track of the exits. Basically for each command that isn't the first one, you will want to send exit to close that inner->inner cmd.
I want to extract the framesize of a video from a file. For this purpose, I have launched an ffmpeg command via bash shell, and I want to extract the output. This command is working well in the bash shell, and returns the output as wanted.
ffprobe -v error -count_frames -of flat=s=_ -select_streams v:0 -show_entries stream=nb_read_frames /home/peter/DA/videos/IMG-2014-1-10-10-4-37.avi
I want to call it via C++ and read out the result. I use the IDE Qt 4.8.6 with GCC 4.8 compiler.
For my code, I use this template:
executing shell command with popen
and changed it for my demands to
#include <iostream>
#include <string>
#include <stdio.h>
using namespace std;
int main()
{
FILE* pipe = popen("echo $(ffprobe -v error -count_frames -of flat=s=_ -select_streams v:0 -show_entries stream=nb_read_frames /home/peter/DA/videos/IMG-2014-1-10-10-4-37.avi)", "r");
if(!pipe)
{
cout << "error" << endl;
return 1;
}
char* buffer = new char[512];
string result;
fgets(buffer, sizeof(buffer), pipe) ;
while(!feof(pipe))
{
if(fgets(buffer, sizeof(buffer), pipe) != NULL)
{
cout << buffer << endl;
result += buffer;
}
}
pclose(pipe);
cout << result<< endl;
return 0;
}
The Qt console returned me this warning, and it is rending with return 0:
/home/peter/DA/videos/IMG-2014-1-10-10-4-37.avi: Invalid data found when processing input
and "pipe" is empty.
When I compile the main.cpp file above with g++ in the shell it works nice too.
Old post, but as I see, there are two points here:
Error "Invalid data found when processing input"
That's an ffprobe normal file processing error. Usually it happens when there are errors inside media file, it is not related to c++ program.
ffprobe writes warning/error messages into stderr stream, but popen only captures stdout stream, that's why your program couldn't get that error message trough the pipe.
How get the stdout+stderr in my program
popen allows execute any shell command, so we can use it to redirect stderr into stdout, so your program can get that output too, like this:
FILE *pipe = popen("ffprobe ... 2>&1");
The 2> redirect handle#2 output into current &1 handle#1 output (#1=stdout, #2=stderr).
There's absolute no need to execute FILE *pipe = popen("echo $(ffprobe ...)");, because the final result will be the same: Note that $(...) returns a string with stdout command output, and echo prints it. Totally redundant.
A few observations in order to improve your code:
When a string is too big to be displayed in one screen width, it's better split it into multiple lines (maybe grouping text inside each line within some logic), because that will improve the reading of your code by other people (and eventually by yourself in a few months).
You can do this with a C/C++ compiler feature that concatenates strings separated by spaces (newlines, tab, etc.), ex. "hi " "world" is the same as "hi world" to the compiler.
When your program have to write error messages, use the stderr stream. In c++ that's std::cerr instead std::cout.
Always free memory allocated when it's no loger used (each new has to have a delete)
Avoid use using namespace std;, instead use using std::name; for each standard instance/class that you'll use. Ex. using std::string;, that avoids future problems, specially in big programs. An example of a common error is here. In general avoid using using namespace xxxx;.
Reorganizing your code, we have:
#include <iostream>
#include <stdio.h>
using std::string;
using std::cout;
using std::cerr;
using std::endl;
int main() {
static char ffprobeCmd[] =
"ffprobe " // command
"-v error " // args
"-count_frames "
"-of flat=s=_ "
"-select_streams v:0 "
"-show_entries stream=nb_read_frames "
"/home/peter/DA/videos/IMG-2014-1-10-10-4-37.avi" // file
" 2>&1"; // Send stderr to stdout
FILE *pipe = popen(ffprobeCmd, "r");
if (!pipe) {
perror("Cannot open pipe.");
return 1;
}
char* buffer = new char[512];
string result;
while ((fgets(buffer, sizeof(buffer), pipe)) != NULL) {
result += buffer;
}
// See Note below
int retCode = pclose(pipe);
if (retCode != 0) {
// Program ends with error, and result has the error message
cerr << "retCode: " << retCode << "\nMessage: " << result << endl;
return retCode;
} else {
// Program ends normally, prints: streams_stream_0_nb_read_frames="xxx"
cout << result << endl;
}
delete buffer; // free memory
return 0;
}
Note
pclose is not intended to return the executed program status code, but if you need this value, pclose does it in some c++ versions/systems, so check it. Anyway it will be zero only if everything was OK.
In my code, I use popen with cmd 'iostat -xk 2 2'. Generally the iostat cmd will output a report to stdout and sleep for interval 2 seconds, then output another report to stdout.
When I use fgets to read from popen pipe, weird things are: I only get the first report. fgets hangs there and I never get the second report 2 seconds later. I change the function fgets to fgetc but it didn't help.
Here is my C++ code:
string iostatCMD = string("iostat -xk 2 2 ");
FILE *iostatFile = popen(iostatCMD.c_str(), "r");
int fd = fileno(iostatFile);
string readall;
while(!feof(iostatFile))
{
char buffer[128];
if (fgets(buffer, 128, iostatFile) != NULL)
{
cout <<buffer;
readall += buffer;
}
}
cout <<"readall = " <<readall <<endl;
I give it a try with Python. It works very well with popen, I get two reports by iostat and no hangs:
subprocess.Popen('iostat -xk 2 2', shell=True, stdout=subprocess.PIPE).communicate()[0];
So my question is how to read continuely all the output from popen pipe in C++?
I am developing an application, in which I want to redirect the output (progress information) of dd command to my C++ program, but it is not actually getting the output, here is the code
FILE * progressInfo = popen("gzip -dc backup/backup.img.gz | pv -ptrbe -i 2 -s 2339876653 | dd of=/dev/sdb","r");
if(!progressInfo)
{
return -1;
}
char buf[1024];
while(fgets(buff, sizeof(buff),progressInfo)!=NULL)
{
std::cout << buff << endl;
}
but the problem is the progress information is not received in buff, and the output is continuously printed on terminal, and above program halts on while(fgets(buff, sizeof(buff),progressInfo)!=NULL), and as soon as the dd operation is completed, the very next line to loop block is executed.
if anyone has any idea why the output is not returned to buff, and its continuously retuned on terminal?
The output is probably being written to standard error rather than standard output. Just add " 2>&1" to the very end of your command string and you should see the output (note the leading space).