I want to be able to read and write to a program from C++. It seems like pstream can do the job, but I find the documentation difficult to understand and have not yet find an example.
I have setup the following minimum working example. This opens python, which in turn (1) prints hello (2) ask input, and (3) prints hello2:
#include <iostream>
#include <cstdio>
#include "pstream.h"
using namespace std;
int main(){
std::cout << "start";
redi::pstream proc(R"(python -c "if 1:
print 'hello'
raw_input()
print 'hello2'
")");
std::string line;
//std::cout.flush();
while (std::getline(proc.out(), line)){
std::cout << " " << "stdout: " << line << '\n';
}
std::cout << "end";
return 0;
}
If I run this with the "ask input" part commented out (i.e. #raw_input()), I get as output:
start stdout: hello
stdout: hello2
end
But if I leave the "ask input" part in (i.e. uncommented raw_input()), all I get is blank, not even start, but rather what seems like a program waiting for input.
My question is, how can one interact with this pstream, how can one establish a little read-write-read-write session? Why does the program not even show start or the first hello?
EDIT:
I don't seem to be making much progress. I don't think I really grasp what is going on. Here are some further attempts with commentary.
1) It seems like I can successfully feed raw_inputI prove this by writing to the child's stderr:
int main(){
cout << "start" <<endl;
redi::pstream proc(R"(python -c "if 1:
import sys
print 'hello'
sys.stdout.flush()
a = raw_input()
sys.stdin.flush()
sys.stderr.write('hello2 '+ a)
sys.stderr.flush()
")");
string line;
getline(proc.out(), line);
cout << line << endl;
proc.write("foo",3).flush();
cout << "end" << endl;
return 0;
}
output:
start
hello
end
hello2 foo
But it locks if I try to read from the stdout again
int main(){
...
a = raw_input()
sys.stdin.flush()
print 'hello2', a
sys.stdout.flush()
")");
...
proc.write("foo",3).flush();
std::getline(proc.out(), line);
cout << line << endl;
...
}
output
start
hello
2) I can't get the readsome approach to work at all
int main(){
cout << "start" <<endl;
redi::pstream proc(R"(python -c "if 1:
import sys
print 'hello'
sys.stdout.flush()
a = raw_input()
sys.stdin.flush()
")");
std::streamsize n;
char buf[1024];
while ((n = proc.out().readsome(buf, sizeof(buf))) > 0)
std::cout.write(buf, n).flush();
proc.write("foo",3).flush();
cout << "end" << endl;
return 0;
}
output
start
end
Traceback (most recent call last):
File "<string>", line 5, in <module>
IOError: [Errno 32] Broken pipe
The output contains a Python error, it seems like the C++ program finished while the Python pipe were still open.
Question: Can anyone provide a working example of how this sequential communication should be coded?
But if I leave the "ask input" part in (i.e. uncommented raw_input()), all I get is blank, not even start, but rather what seems like a program waiting for input.
The Python process is waiting for input, from its stdin, which is connected to a pipe in your C++ program. If you don't write to the pstream then the Python process will never receive anything.
The reason you don't see "start" is that Python thinks it's not connected to a terminal, so it doesn't bother flushing every write to stdout. Try import sys and then sys.stdout.flush() after printing in the Python program. If you need it to be interactive then you need to flush regularly, or set stdout to non-buffered mode (I don't know how to do that in Python).
You should also be aware that just using getline in a loop will block waiting for more input, and if the Python process is also blocking waiting for input you have a deadlock. See the usage example on the pstreams home page showing how to use readsome() for non-blocking reads. That will allow you to read as much as is available, process it, then send a response back to the child process, so that it produces more output.
EDIT:
I don't think I really grasp what is going on.
Your problems are not really problems with pstreams or python, you're just not thinking through the interactions between two communicating processes and what each is waiting for.
Get a pen and paper and draw state diagrams or some kind of chart that shows where the two processes have got to, and what they are waiting for.
1) It seems like I can successfully feed raw_input
Yes, but you're doing it wrong. raw_input() reads a line, you aren't writing a line, you're writing three characters, "foo". That's not a line.
That means the python process keeps trying to read from its stdin. The parent C++ process writes the three characters then exits, running the pstream destructor which closes the pipes. Closing the pipes causes the Python process so get EOF, so it stops reading (after only getting three characters not a whole line). The Python process then prints to stderr, which is connected to your terminal, because you didn't tell the pstream to attach a pipe to the child's stderr, and so you see that output.
But it locks if I try to read from the stdout again
Because now the parent C++ process doesn't exit, so doesn't close the pipes, so the child Python process doesn't read EOF and keeps waiting for more input. The parent C++ process is also waiting for input, but that will never come.
If you want to send a line to be read by raw_input() then write a newline!
This works fine, because it sends a newline, which causes the Python process to get past the raw_input() line:
cout << "start" <<endl;
redi::pstream proc(R"(python -c "if 1:
import sys
print 'hello'
sys.stdout.flush()
a = raw_input()
print 'hello2', a
sys.stdout.flush()
")");
string line;
getline(proc, line);
cout << line << endl;
proc << "foo" << endl; // write to child FOLLOWED BY NEWLINE!
std::getline(proc, line); // read child's response
cout << line << endl;
cout << "end" << endl;
N.B. you don't need to use proc.out() because you haven't attached a pipe to the process' stderr, so it always reads from proc.out(). You would only need to use that when reading from both stdout and stderr, where you would use proc.out() and proc.err() to distinguish them.
2) I can't get the readsome approach to work at all
Again, you have the same problem that you're only writing three characters and so the Python processes waits forever. The C++ process is trying to read as well, so it also waits forever. Deadlock.
If you fix that by sending a newline (as shown above) you have another problem: the C++ program will run so fast that it will get to the while loop that calls readsome before the Python process has even started. It will find nothing to read in the pipe, and so the first readsome call returns 0 and you exit the loop. Then the C++ program gets to the second while loop, and the child python process still hasn't started printing anything yet, so that loop also reads nothing and exits. Then the whole C++ program exits, and finally the Python child is ready to run and tries to print "hello" but by then it's parent is gone and it can't write to the pipe.
You need readsome to keep trying if there's nothing to read the first time you call it_, so it waits long enough for the first data to be readable.
For your simple program you don't really need readsome because the Python process only writes a single line at a time, so you can just read it with getline. But if it might write more than one line you need to be able to keep reading until there's no more data coming, which readsome can do (it reads only if there's data available). But you also need some way to tell whether more data is still going to come (maybe the child is busy doing some calculations before it sends more data) or if it's really finished. There's no general way to know that, it depends on what the child process is doing. Maybe you need the child to send some sentinel value, like "---END OF RESPONSE---", which the parent can look for to know when to stop trying to read more.
For the purposes of your simple example, let's just assume that if readsome gets more than 4 bytes it received the whole response:
cout << "start" <<endl;
redi::pstream proc(R"(python -c "if 1:
import sys
print 'hello'
sys.stdout.flush()
a = raw_input()
sys.stdin.flush()
print 'hello2', a
sys.stdout.flush()
")");
string reply;
streamsize n;
char buf[1024];
while ((n = proc.readsome(buf, sizeof(buf))) != -1)
{
if (n > 0)
reply.append(buf, n);
else
{
// Didn't read anything. Is that a problem?
// Need to try to process the content of 'reply' and see if
// it's what we're expecting, or if it seems to be incomplete.
//
// Let's assume that if we've already read more than 4 characters
// it's a complete response and there's no more to come:
if (reply.length() > 3)
break;
}
}
cout << reply << std::flush;
proc << "foo" << std::endl;
while (getline(proc, reply)) // maybe use readsome again here
cout << reply << std::endl;
cout << "end" << endl;
This loops while readsome() != -1, so it keeps retrying if it reads nothing and only stops the loop if there's an error. In the loop body it decides what to do if nothing was read. You'll need to insert your own logic in here that makes sense for whatever you're trying to do, but basically if readsome() hasn't read anything yet, then you should loop and retry. That makes the C++ program wait long enough for the Python program to print something.
You'd probably want to split out the while loop into a separate function that reads a whole reply into a std::string and returns it, so that you can re-use that function every time you want to read a response. If the child sends some sentinel value that function would be easy to write, as it would just stop every time it receives the sentinel string.
Related
I have a task to create a program that does some calculation on a vector of numbers. The vector must contain at least 1 number in it and if it doesn't I have to throw an exception and try again. There is a video example how the code should work here: https://asciinema.org/a/283343
I'm guessing that EOF is being signaled using CTRL+D and that's what causes the exception to be thrown.
If they were using Enter (new line), it would leave a blank line behind.
But in my case, after I press CTRL+D, my program just runs in an infinite loop because the stdin stays in a failed state despite me using cin.clear().
Is there another shortcut similar to CTRL+D that they might be using for this, or is there a way to reopen the stream, or restart the whole application.
The program runs fine on Windows when I use CTRL+Z, but on Linux I just can't get it to work the same.
Example code below:
#include<iostream>
#include<vector>
void enter_elements(std::vector<double>& input_list){
double x;
std::cout << "Enter numbers: " << std::endl;
while(std::cin >> x){
input_list.push_back(x);
}
if(input_list.empty()){
throw std::string("You must enter at least 1 number!");
}
}
int main(){
std::vector<double> input_list;
try {
enter_elements(input_list);
} catch (const std::string& e) {
std::cout << "Error: " << e << std::endl;
std::cin.clear();
enter_elements(input_list);
}
return 0;
}
Can anyone help me with this problem or suggest where could I maybe read more about it?
No.
Once the stream is closed, the stream is closed. That's it.
What I'd do is accept a set of numbers on one line. Your input iteration would end at the end of the line. Then you validate those numbers, and ask for another line if necessary.
You can do that by looping over std::getline instead of using formatted extraction. Then you'd need to parse the line you get.
That's not what the video shows, but I don't know how they achieved that. Maybe you should ask them!
Are you asking how to stop the program?
If so, on linux, you should try CTRL+C instead of CTRL+D or CTRL+Z while your program is running
This is the sort of reason why I hate using cin >> anInt. I much prefer to getLine and parse it myself. Then you can make it do anything and decide how you're going to determine end of list, etc.
Yes, it's more code. But you don't run into these weird problems like this.
Just started C++ learning by C++ Primer 5th ed.
The very first example of the book on page 6 is as followed
#include <iostream>
int main()
{
std::cout << "Enter two numbers:" << std::endl;
int v1 = 0, v2 = 0;
std::cin >> v1 >> v2;
std::cout << "The sum of " << v1 << " and " << v2
<< " is " << v1 + v2 << std::endl;
return 0;
}
The manipulator endl insert newline and flush the buffer.
soon followed by the code snipe in the page 7, the author emphasized that
Programmers often add print statements during debugging. Such
statements should always flush the stream. Otherwise, if the program
crashes, output may be left in the buffer, leading to incorrect
inference about where the program crashed
From the code example and the emphasized warning I feel it is important to do the flush when writing to a Stream
Here is the part I don't understand, how a bout reading from a stream case, such as std::cin, is there any necessary to do the flush then?
Appended Question:
#include <iostream>
int main()
{
int sum = 0, val = 1;
while (val <= 5) {
sum += val;
++val;
std::cout << sum << std::endl; //test
}
}
When I changed the line marked with test to std::cout << sum << '\n';, there is no visual difference in the console. Why is that, isn't it supposed to print as follows if there is no flush for each loop?
1
1 3
1 3 6
1 3 6 10
1 3 6 10 15
Thanks
Even if #Klitos Kyriacou is right when he says you should create a new post for a new question, it seems to me that both your questions arise from the same misunderstanding.
Programmers often add print statements during debugging. Such statements should always flush the stream. Otherwise, if the program crashes, output may be left in the buffer, leading to incorrect inference about where the program crashed
This quote does not mean that you need to flush every buffer in your program to create any output on the console.
By flushing the buffers you can make sure that the output is printed before the next line of code is executed.
If you don't flush the buffers and your program finishes, the buffers will be flushed anyway.
So, the reason you see the same output on the console with std::endl and \n is, that exactly the same text is printed to the console. In the former case, the output might be there slightly earlier, as the buffers are flushed early. In the latter case, the buffers are flushed later, but they will be flushed at some time.
What the quote talks about is the case when your program does not exit gracefully, e.g. when your program crashes or is interrupted by the OS. In these cases, your output might not be written to the console when you did not explicitly flush the buffers.
What the quote wants you to know is: Whenever you want to debug a program that crashes, you should explicitly flush the buffers to make sure your debug output is printed to the console before your program is interrupted.
Note that this might not be true for all implementations.
From http://en.cppreference.com/w/cpp/io/manip/endl
In many implementations, standard output is line-buffered, and writing '\n' causes a flush anyway, unless std::ios::sync_with_stdio(false) was executed.
Quote from the same book page 26:
Buffer A region of storage used to hold data. IO facilities often store input (or out-put) in a buffer and read or write the buffer
independently from actions in the program. Output buffers can be
explicitly flushed to force the buffer to be written. Be default,
reading cin flushes cout; cout is also flushed when the program ends
normally.
This question already has answers here:
endl and flushing the buffer
(5 answers)
Closed 6 years ago.
std::cout << "Enter two numbers:";
std::cout << std:endl;
This code snippet is followed by two paragraphs and a warning note, among which I understood the first para, but neither the second one nor the note. The text is as follows -
"The first output operator prints a message to the user. That message
is a string literal, which is a sequence of characters enclosed in
double quotation marks. The text between the quotation marks is
printed to the standard output.
The second operator prints endl,
which is a special value called a manipulator. Writing endl has
the effect of ending the current line and flushing the buffer
associated with that device. Flushing the buffer ensures that all the
output the program has generated so far is actually written to the
output stream, rather than sitting in memory waiting to be written.
Warning Programmers often add print statements during debugging. Such statement should always flush the stream. Otherwise, if the
program crashes, output may be left in the buffer, leading to
incorrect inferences about where the program crashed."
So I didn't understand of the part of endl, nor the following warning. Can anyone please explain this to me as explicitly as possible and please try to keep it simple.
Imagine you have some code that crashes somewhere, and you don't know where. So you insert some print statements to narrow the problem down:
std::cout << "Before everything\n";
f1();
std::cout << "f1 done, now running f2\n";
f2();
std::cout << "all done\n";
Assuming that the program crashes during the evaluation of either f1() or f2(), you may not see any output, or you may see partial output that is misleading -- e.g. you could see only "Before everything", even though the crash happened in f2(). That's because the output data may be waiting in a buffer and hasn't actually been written to the output device.
The Primer's recommendation is therefore to flush each output, which you can conveniently achieve with endl:
std::cout << "Before everything" << std::endl;
f1();
std::cout << "f1 done, now running f2" << std::endl;
f2();
std::cout << "all done" << std::endl;
An alternative is to write debug output to std::cerr instead, which is not buffered by default (though you can always change the buffering of any ostream object later).
A more realistic use case is when you want to print a progress bar in a loop. Usually, a newline (\n) causes line-based output to be printed anyway, but if you want to print a single character for progress, you may not see it printed at all until after all the work is done unless you flush:
for (int i = 0; i != N; ++i)
{
if (i % 1000 == 0)
{
std::cout << '#'; // progress marger
std::cout.flush();
}
do_work();
}
std::cout << '\n';
Well, simply:
std::cout << "Hello world!";
will print "Hello world!" and will remain in the same line. Now if you want to go to a new line, you should use:
std::cout << "\n";
or
std::cout << std::endl;
Now before I explain the difference, you have to know 1 more simple thing: When you issue a print command with the std::cout stream, things are not printed immediately. They are stored in a buffer, and at some point this buffer is flushed, either when the buffer is full, or when you force it to flush.
The first kind, \n, will not flush, but the second kind std::endl, will go to a new line + flush.
Operating systems do buffered IO. That is, when your program outputs something, they dont necessarily put it immediately where it should go (i.e. disk, or the terminal), they might decide to keep the data in an internal memory buffer for some while before performing the actual IO operation on the device.
They do this to optmize performance, because doing the IO in chunks is better than doing it immediately as soon as there are a few bytes to write.
Flushing a buffer means asking the OS to perform immediately the IO operation without any more waiting. A programmer would do this this when (s)he knows that waiting for more data doesn't make sense.
The second note says that endl not only prints a newline, but also hints the cout to flush its buffer.
The 3rd note warns that debugging errors, if buffered and not flushed immediately, might not be seen if the program crashes while the error messages are still in the buffer (not flushed yet).
Does anyone know how to make an event loop in c++ without a library? It doesn't have to be cross-platform, I'm on a Mac. Basically, I want the program to run and do nothing until the user presses the up arrow key, then the program will output "You pressed up" or something. All i can think of is having an infinite while or for loop and get input with cin, but I don't think cin can detect arrow keys and I believe it pauses the program until it reaches a '\n';
I would want it to look like this:
void RUN()
{
while(true)
{
// poll events and do something if needed
}
}
int main()
{
RUN();
}
I'm kinda sure it's possible without threads, and I've heard that this can be accomplished with fd_set or something, but I'm not sure how.
Any help would be really appreciated.
EDIT:
The program has to run in the background when there aren't any events. For example, Microsoft Word doesn't stop until the user presses a button, it keeps running. I want something like that, but command-line not GUI.
Since you're talking keyboard input, and not looking for a Mac look and feel, what you want is the UNIX way of doing it. And that is,
1) set the terminal in either raw or cbrk mode (I forget which).
2) now use read() to read single characters at a time.
3) temporarily echo the character read (as an int) so you can find what the up arrow key gives you.
As for the more general event loop question, where the only input device is the keyboard, you sit in a loop, and whenever a key is typed (in raw mode?) you call a routine with the value of the key typed. If you had more input devices, you would need multiple threads each could listen to a different device, putting what they find on a queues (with appropriate locking). The main loop would then check the queue and call a routine appropriately everytime something appears in it.
You can use ncurses and enable cbreak to get the raw input stream.
I've used a while loop with signal handlers. Like this incomplete snippet.
void getSomething()
{
std::cout << "Enter new step size: "; std::cout.flush();
std::cin >> globalVariable;
std::getchar(); // consume enter key.
}
void printCommands()
{
std::cout << "1: do something\n"
<< "q: quit\n"
<< "h: help\n"
<< std::endl;
}
void getCommand()
{
// Output prompt
std::cout << "Enter command ('h' for help): "; std::cout.flush();
// Set terminal to raw mode
int ret = system("stty raw");
// Wait for single character
char input = std::getchar();
// Reset terminal to normal "cooked" mode
ret = system("stty cooked");
std::cout << std::endl;
if (input == 'h') printCommands();
else if (input == '1') getSomething();
else if (input == 'q') {
g_next = true;
g_quit = true;
}
}
void
signalHandler(int signo)
{
if (signo == SIGINT) {
g_next = true;
} else if (signo == SIGQUIT) {
getCommand();
}
}
int main(int argc, char* argv[])
{
signal(SIGINT, signalHandler);
signal(SIGUSR1, signalHandler);
signal(SIGQUIT, signalHandler);
do {
// Stuff
} while (!g_quit);
exit(0);
}
The question has been updated to say "The program has to run in the background ... but command-line not GUI."
All traditional; *NIX shells that can put a program into the background also disconnect the program's standard input from the terminal, so AFAIK, this has become impossible.
This does not need to be Mac specific. The Mac supports *NIX mechanisms for reading characters from a keyboard.
AFAICT all the program is doing is waiting for a character, so it might as well block.
Normally the terminal device, tty (teletype!), is interpreting characters typed on the keyboard before your program can read them from standard input. Specifically the tty device normally buffers an entire line of text, and intercepts the rubout character (and a few others like CTRL+w) to edit the line of text. This pre-processing of characters is called a 'line discipline'
You need to set the tty device driver to stop doing that! Then you can get all of the characters the user types.
You change the device using ioctl or termios on the file descriptor.
Search for e.g. "ioctl tty line discipline raw" to understand the details, and find program examples.
You can set the terminal to 'raw' using the command line program stty.
Please read the stty man page because setting it back can be slightly tricky (NB: if you make a mistake it is often easier to kill the terminal, than try to fix it, because there is not echoing of anything you type)
It is possible that the up-arrow is not a single char, so it will require some byte-at-a-time decoding to avoid blocking at the wrong point in the input stream, i.e. if some input sequences are one character, and others two, or three characters, the decoding needs to happen at each byte to decide if there is a pending byte, or one too many read's might get issued, which would cause the program to block.
I just started learning C++ and I wrote this sample program from the text and when I compile and run it, it just closes after the user inputs any number and presses enter. I'm guessing the answer to this is very obvious so forgive me as newbie here....it's really my first C++ program :P
#include <iostream>
using namespace std;
int main ()
{
int numberOfLanguages;
cout << "Hello Reader.\n"
<< "Welcome to C++.\n"
cout << "How many programming languages have you used? ";
cin >> numberOfLanguages;
if(numberOfLanguages < 1)
cout << "Read the preface. You may prefer.\n"
<< "a more elementary book by the same author.\n";
else
cout << "Enjoy the book.\n";
return 0;
}
Imagine you were designing a model for application execution. You have two choices:
A) When the end of a program is reached it shall terminate.
B) When the end of a program is reached the program shall remain alive in some strange limbo state. It will still retain system resources and will not actually be doing anything, but in order to close the user must terminate it explicitly.
I think anyone would go for option A here, and that is what you are seeing. The end of main is reached and your program exits.
If you would like it to pause at the end take some input from the user, i.e.,
char c;
std::cin >> c;
return 0;
The program closes because there's nothing more for the program to do. It outputs the final statements really really fast and then reaches return 0 which causes it to exit. You'll want to do something there to pause the program.
On Windows, a basic way to do that is system("pause"); (you will need #include <stdlib.h>)
cin.getline is a more standard way to do it.
It closes because the execution reaches return 0; and there is nothing left to do.
If you want the program to wait before closing you could add an something like this:
cout << "Press enter to exit...";
cin >> someVarThatWontBeUsed;
You could also run the program from the command line instead of running the .exe. It will reach end of execution anyway but the prompt will stay open.
Your program ends right after you print out your text. If you want to see anything on the screen you can add a cin right before your return 0 so your program waits for a user response before exiting.
// Wait for user to hit enter
cin >> dummyVar;
return 0;
Either put another read from cin to wait for the user, or open the Command Prompt yourself and run it there.
The program you posted has an error. I was not able to compile what you posted.
cout << "Hello Reader.\n"
<< "Welcome to C++.\n"
is not terminated with a semicolon. I added a semicolon and it compiles and runs as you expect.
Edit: Of course, you have to run the program in a terminal that stays open after the program exits, or use cin to wait for more input, or something like that.
After the user inputs a number, which is saved to numberOfLanguages, it reaches return 0 which returns from the main function and thus the program ends.