I got some problems with EOF and stdio in a communication pipeline between a python process and a C++ program. I have no idea what I am doing wrong. When I see an EOF in my program I clear the stdin and next round I try to read in a new line. The problem is: for some reason the getline function immediatly (from the second run always, the first works just as intended) returns an EOF instead of waiting for a new input from the python process... Any idea?
alright Here is the code:
#include <string>
#include <iostream>
#include <iomanip>
#include <limits>
using namespace std;
int main(int argc, char **argv) {
for (;;) {
string buf;
if (getline(cin,buf)) {
if (buf=="q") break;
/*****///do some stuff with input //my actual filter program
cout<<buf;
/*****/
} else {
if ((cin.rdstate() & istream::eofbit)!=0)cout<<"eofbit"<<endl;
if ((cin.rdstate() & istream::failbit)!=0)cout<<"failbit"<<endl;
if ((cin.rdstate() & istream::badbit)!=0)cout<<"badbit"<<endl;
if ((cin.rdstate() & istream::goodbit)!=0)cout<<"goodbit"<<endl;
cin.clear();
cin.ignore(numeric_limits<streamsize>::max());
//break;//I am not using break, because I
//want more input when the parent
//process puts data into stdin;
}
}
return 0;
}
and in python:
from subprocess import Popen, PIPE
import os
from time import sleep
proc=Popen(os.getcwd()+"/Pipingtest",stdout=PIPE,stdin=PIPE,stderr=PIPE);
while(1):
sleep(0.5)
print proc.communicate("1 1 1")
print "running"
communicate in python is a one shot function. It sends the given input to a process, closes the input stream, and reads the output streams, waiting for the process to terminate.
There is no way you can 'restart' the pipe with the same process after "communicating".
Conversely, on the other side of the pipe, when you read EOF there is no more data to read. Any attempt to read will immediately return EOF; python has closed the pipe.
If you want to carry on communicating with the same pipe you need to use the subprocess' stdin and stdout members and not communicate (but be careful of the potential of deadlocks) and use something other than the end of stream to signal that the C++ side should do another "batch" of processing.
Related
Using a c++ program i can successfully send commands to an arduino. The code uses the command:
system("$echo [command] > dev/ttyACM0");
Currently i must manually input the commands into this space, I was wondering if it's possible for a user to input the command, and for it to then be added to the string within system()?
This is an approximation of what I think you want:
#include <fstream>
#include <iostream>
#include <string>
int main() {
std::string command;
if(std::getline(std::cin, command)) { // read user input
std::ofstream ard("/dev/ttyACM0"); // open the device
if(ard) {
ard << command << '\n'; // send the command
}
} // here `ard` goes out of scope and is closed automatically
}
Note that you do not need the unsafe system() command at all here. Just open the device and send the string directly.
I am trying to read in a string of length in 10^5 order. I get incorrect string if the size of string grows beyond 4096.
I am using the following code
string a;
cin>>a;
This didn't work then I tried reading character by character by following code
unsigned char c;
vector<unsigned char> a;
while(count>0){
c = getchar();
a.push_back(c);
count--;
}
I have done necessary escaping for using getchar this also had the 4096 bytes problem. Can someone suggest a workaround or point to correct way of reading it.
It is because your terminal inputs are buffered in the I/O queue of the kernel.
Input and output queues of a terminal device implement a form of buffering within the kernel independent of the buffering implemented by I/O streams.
The terminal input queue is also sometimes referred to as its typeahead buffer. It holds the characters that have been received from the terminal but not yet read by any process.
The size of the input queue is described by the MAX_INPUT and _POSIX_MAX_INPUT parameters;
By default, your terminal is in Canonical mode.
In canonical mode, all input stays in the queue until a newline character is received, so the terminal input queue can fill up when you type a very long line.
We can change the input mode of terminal from canonical mode to non-canonical mode.
You can do it from terminal:
$ stty -icanon (change the input mode to non-canonical)
$ ./a.out (run your program)
$ stty icanon (change it back to canonical)
Or you can also do it programatically,
To change the input mode programatically we have to use low level terminal interface.
So you can do something like:
#include <iostream>
#include <string>
#include <stdio.h>
#include <termios.h>
#include <unistd.h>
int clear_icanon(void)
{
struct termios settings;
int result;
result = tcgetattr (STDIN_FILENO, &settings);
if (result < 0)
{
perror ("error in tcgetattr");
return 0;
}
settings.c_lflag &= ~ICANON;
result = tcsetattr (STDIN_FILENO, TCSANOW, &settings);
if (result < 0)
{
perror ("error in tcsetattr");
return 0;
}
return 1;
}
int main()
{
clear_icanon(); // Changes terminal from canonical mode to non canonical mode.
std::string a;
std::cin >> a;
std::cout << a.length() << std::endl;
}
Using this test-program based on what you posted:
#include <iostream>
#include <string>
int main()
{
std::string a;
std::cin >> a;
std::cout << a.length() << std::endl;
}
I can do:
./a.out < fact100000.txt
and get the output:
456574
However, if I copy'n'paste from an editor to the console, it stops at 4095. I expect that's a limit somewhere in the consoles copy'n'paste handling. The easy solution to that is of course to not use copy'n'paste, but redirect from a file. On some other systems, the restruction to 4KB of input may of course reside somewhere else. (Note that, at least on my system, I can happily copy and paste the 450KB of factorial result to another editor window, so in my system it's simply the console buffer that is the problem).
This is much more likely to be a platform/OS problem than a C++ problem. What OS are you using, and what method are you using to get the string fed to stdin? It's pretty common for command-line arguments to be capped at a certain size.
In particular, given that you've tried reading one character at a time, and it still didn't work, this seems like a problem with getting the string to the program, rather than a C++ issue.
I'm using gcc 7.3 and g++ 7.3. GCC and G++ makes error. For example,
#include <stdio.h>
#include <unistd.h>
int main() {
printf("a");
sleep(1);
return 0;
}
'a' prints after waiting 1 seconds but when I use printf("a\n"); it works correctly. It's same on C++. For example,
#include <iostream>
#include <unistd.h>
int main() {
std::cout << "a";
sleep(1);
return 0;
}
'a' prints after waiting 1 seconds, too. However, when I use std::cout << "a" << std::endl; it works correctly. What's the problem and how to fix it?
sleep() is like schedule a process manually. printf() puts the data into stdout stream not directly on monitor.
printf("a"); /* data is there in stdout , not flushed */
sleep(1); /* as soon as sleep(1) statement occurs your process(a.out) jumped to waiting state, so data not gets printed on screen */
So either you should use fflush(stdout) or use \n to clear the stdout stream.
You are seeing this behaviour because stdout will be usually line buffered when used with terminal and fully buffered when used with files, the strings will be stored in a buffer and can be flushed by entering new line or when buffer fills or when program terminates
You can also override buffer mode by using setvbuf as below
setvbuf(stdout, NULL, _IONBUF, 1024);
printf("a");
It will print a without buffering, have a look at https://www.tutorialspoint.com/c_standard_library/c_function_setvbuf.htm for using setvbuf
Also have a look at different types of buffering with streams.
Hope this helps you.
Say I have an .exe, lets say sum.exe. Now say the code for sum.exe is
void main ()
{
int a,b;
scanf ("%d%d", &a, &b);
printf ("%d", a+b);
}
I wanted to know how I could run this program from another c/c++ program and pass input via stdin like they do in online compiler sites like ideone where I type the code in and provide the stdin data in a textbox and that data is accepted by the program using scanf or cin. Also, I wanted to know if there was any way to read the output of this program from the original program that started it.
The easiest way I know for doing this is by using the popen() function. It works in Windows and UNIX. On the other way, popen() only allows unidirectional communication.
For example, to pass information to sum.exe (although you won't be able to read back the result), you can do this:
#include <stdio.h>
#include <stdlib.h>
int main()
{
FILE *f;
f = popen ("sum.exe", "w");
if (!f)
{
perror ("popen");
exit(1);
}
printf ("Sending 3 and 4 to sum.exe...\n");
fprintf (f, "%d\n%d\n", 3, 4);
pclose (f);
return 0;
}
In C on platforms whose name end with X (i.e. not Windows), the key components are:
pipe - Returns a pair of file descriptors, so that what's written to one can be read from the other.
fork - Forks the process to two, both keep running the same code.
dup2 - Renumbers file descriptors. With this, you can take one end of a pipe and turn it into stdin or stdout.
exec - Stop running the current program, start running another, in the same process.
Combine them all, and you can get what you asked for.
This is my solution and it worked:
sum.cpp
#include "stdio.h"
int main (){
int a,b;
scanf ("%d%d", &a, &b);
printf ("%d", a+b);
return 0;
}
test.cpp
#include <stdio.h>
#include <stdlib.h>
int main(){
system("./sum.exe < data.txt");
return 0;
}
data.txt
3 4
Try this solution :)
How to do so is platform dependent.
Under windows, Use CreatePipe and CreateProcess. You can find example from MSDN :
http://msdn.microsoft.com/en-us/library/windows/desktop/ms682499(v=vs.85).aspx
Under Linux/Unix, you can use dup() / dup2()
One simple way to do so is to use a Terminal (like command prompt in windows) and use | to redirect input/output.
Example:
program1 | program2
This will redirect program1's output to program2's input.
To retrieve/input date, you can use temporary files, If you don't want to use temporary files, you will have to use pipe.
For Windows, (use command prompt):
program1 <input >output
For Linux, you can use tee utility, you can find detail instruction by typing man tee in linux terminal
It sounds like you're coming from a Windows environment, so this might not be the answer you are looking for, but from the command line you can use the pipe redirection operator '|' to redirect the stdout of one program to the stdin of another. http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/redirection.mspx?mfr=true
You're probably better off working in a bash shell, which you can get on Windows with cygwin http://cygwin.com/
Also, your example looks like a mix of C++ and C, and the declaration of main isn't exactly an accepted standard for either.
How to do this (you have to check for errors ie. pipe()==-1, dup()!=0, etc, I'm not doing this in the following snippet).
This code runs your program "sum", writes "2 3" to it, and than reads sum's output. Next, it writes the output on the stdout.
#include <iostream>
#include <sys/wait.h>
#include <unistd.h>
int main() {
int parent_to_child[2], child_to_parent[2];
pipe(parent_to_child);
pipe(child_to_parent);
char name[] = "sum";
char *args[] = {name, NULL};
switch (fork()) {
case 0:
// replace stdin with reading from parent
close(fileno(stdin));
dup(parent_to_child[0]);
close(parent_to_child[0]);
// replace stdout with writing to parent
close(fileno(stdout));
dup(child_to_parent[1]);
close(child_to_parent[1]);
close(parent_to_child[1]); // dont write on this pipe
close(child_to_parent[0]); // dont read from this pipe
execvp("./sum", args);
break;
default:
char msg[] = "2 3\n";
close(parent_to_child[0]); // dont read from this pipe
close(child_to_parent[1]); // dont write on this pipe
write(parent_to_child[1], msg, sizeof(msg));
close(parent_to_child[1]);
char res[64];
wait(0);
read(child_to_parent[0], res, 64);
printf("%s", res);
exit(0);
}
}
I'm doing what #ugoren suggested in their answer:
Create two pipes for communication between processes
Fork
Replace stdin, and stdout with pipes' ends using dup
Send the data through the pipe
Based on a few answers posted above and various tutorials/manuals, I just did this in Linux using pipe() and shell redirection. The strategy is to first create a pipe, call another program and redirect the output of the callee from stdout to one end of the pipe, and then read the other end of the pipe. As long as the callee writes to stdout there's no need to modify it.
In my application, I needed to read a math expression input from the user, call a standalone calculator and retrieve its answer. Here's my simplified solution to demonstrate the redirection:
#include <string>
#include <unistd.h>
#include <sstream>
#include <iostream>
// this function is used to wait on the pipe input and clear input buffer after each read
std::string pipeRead(int fd) {
char data[100];
ssize_t size = 0;
while (size == 0) {
size = read(fd, data, 100);
}
std::string ret = data;
return ret;
}
int main() {
// create pipe
int calculatorPipe[2];
if(pipe(calculatorPipe) < 0) {
exit(1);
}
std::string answer = "";
std::stringstream call;
// redirect calculator's output from stdout to one end of the pipe and execute
// e.g. ./myCalculator 1+1 >&8
call << "./myCalculator 1+1 >&" << calculatorPipe[1];
system(call.str().c_str());
// now read the other end of the pipe
answer = pipeRead(calculatorPipe[0]);
std::cout << "pipe data " << answer << "\n";
return 0;
}
Obviously there are other solutions out there but this is what I can think of without modifying the callee program. Things might be different in Windows though.
Some useful links:
https://www.geeksforgeeks.org/pipe-system-call/
https://www.gnu.org/software/bash/manual/html_node/Redirections.html
Is it possible to send data to another C++ program, without being able to modify the other program (since a few people seem to be missing this important restriction)? If so, how would you do it? My current method involves creating a temporary file and starting the other program with the filename as a parameter. The only problem is that this leaves a bunch of temporary files laying around to clean up later, which is not wanted.
Edit: Also, boost is not an option.
Clearly, building a pipe to stdin is the way to go, if the 2nd program supports it. As Fred mentioned in a comment, many programs read stdin if either there is no named file provided, or if - is used as the filename.
If it must take a filename, and you are using Linux, then try this: create a pipe, and pass /dev/fd/<fd-number> or /proc/self/fd/<fd-number> on the command line.
By way of example, here is hello-world 2.0:
#include <string>
#include <sstream>
#include <cstdlib>
#include <cstdio>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
int main () {
int pfd[2];
int rc;
if( pipe(pfd) < 0 ) {
perror("pipe");
return 1;
}
switch(fork()) {
case -1: // Error
perror("fork");
return 1;
case 0: { // Child
// Close the writing end of the pipe
close(pfd[1]);
// Create a filename that refers to reading end of pipe
std::ostringstream path;
path << "/proc/self/fd/" << pfd[0];
// Invoke the subject program. "cat" will do nicely.
execlp("/bin/cat", "cat", path.str().c_str(), (char*)0);
// If we got here, then something went wrong, then execlp failed
perror("exec");
return 1;
}
default: // Parent
// Close the reading end.
close(pfd[0]);
// Write to the pipe. Since "cat" is on the other end, expect to
// see "Hello, world" on your screen.
if (write(pfd[1], "Hello, world\n", 13) != 13)
perror("write");
// Signal "cat" that we are done writing
close(pfd[1]);
// Wait for "cat" to finish its business
if( wait(0) < 0)
perror("wait");
// Everything's okay
return 0;
}
}
You could use sockets. It sounds like both application are on the same host, so you just identify the peers as localhost:portA and localhost:port B. And if you do it this way you can eventually graduate to do network IO. No temp files, no mystery parse errors or file deletions. TCP guarantees packet delivery and guarantees they will be ordered correctly.
So yeah, I would consider creating an synchronous socket server (use asynchronous if you anticipate having tons of peers). One benefit over pipe oriented IPC is that TCP sockets are completely universal. Piping varies dramatically based upon what system you are on (consider Windows named pipes vs implicit and explicit POSIX pipes -> very different).