I have the following code:
int buffer_max_size = 1024;
char* buffer = new char[buffer_max_size]
FILE* cout_file = fdopen(cout_pipe[0], "r");
while (fread( &buffer[0], sizeof(char),sizeof(char)*buffer_max_size, cout_file) != 0 )
{...}
cout_file is of type FILE* and is connected to a binary's stdout.
That binary outputs some text on its std_out at 5 sec intervals.
It seems that fread is blocking until the cout_file contains buffer_max_size bytes.. Is that normal?
I would like to be able to read what is in the pipe right now without blocking.. Is that possible?
If you want non-blocking I/O, use the OS's read and fcntl functions.
The <stdio.h> API (and also <fstream>) perform additional buffering and may automatically retry reads which end early (for example due to being interrupted by a signal), so they cannot be guaranteed to provide non-blocking I/O even if the underlying file descriptor is configured for it.
Not all OSes will use these POSIX names. In that case your options are platform-specific code (for example, on Windows you would be using ReadFile and SetNamedPipeHandleState), or using a wrapper library such as Boost ASIO to abstract the differences. But make sure that your wrapper is designed to expose non-blocking behavior, otherwise it will cause the same pain as <stdio.h>
Related
Apologies if this question is overly broad. I'm new to C++ and trying to understand different stream types and why they matter (or doesn't matter).
I'm learning by coding a simple program that launch a child process, and process the output. I'm following the Boost process synchronous IO example: https://www.boost.org/doc/libs/1_75_0/doc/html/boost_process/tutorial.html#boost_process.tutorial.io.
One of the example can be reduce to this:
#include <boost/process.hpp>
using namespace std;
using namespace boost::process;
int main(int argc, char *argv[]) {
opstream in;
ipstream out;
child c("c++filt", std_out > out, std_in < in);
in << "_ZN5boost7process8tutorialE" << endl;
in.pipe().close(); // This will help c++filt quit, so we don't hang at wait() forever
c.wait();
return 0;
}
My question is:
Why do we have to use a boost opstream? Can I use istringstream instead (besides that it doesn't compile)? Can make it compile with istringstream?
Boost document said:
Boost.process provides the pipestream (ipstream, opstream, pstream) to wrap around the pipe and provide an implementation of the std::istream, std::ostream and std::iostream interface.
Does being a pipe matter, i.e. does pipe have significant implication here?
What Are Processes, How Do They Talk?
Programs interact with their environment in various ways. One set of channels are the standard input, output and error streams.
These are often tied to a terminal or files by a shell (cmd.exe, sh, bash etc).
Now if programs interact with eachother, like:
ls | rev
to list files and send the output to another program (rev, which reverses each line), this is implemented with pipes. Pipes are an operating system feature, not a boost idea. All major operating systems have them.
Fun fact: the | operator used in a most shells to indicate this type of output/input redirection between processes is called the PIPE symbol.
What Is A Pipe, Then?
Pipes are basically "magic" file-descriptors that refer to an "IO channel" rather than a file. Pipes have two ends: One party can writes to one end, the other party reads from the other.
Why?
Two reasons that come to mind right away
Files require disk IO and syncing, making it slow
Another fun fact: MSDOS has implemented pipes in terms of temporary files (on disk) for a very long time:
MS-DOS 2.0 introduced the ability to pipe the output of one program as the input of another. Since MS-DOS was a single-tasking operating system, this was simulated by redirecting the first program’s output to a temporary file and running it to completion, then running the second program with its input redirected from that temporary file. Now all of a sudden, MS-DOS needed a location to create temporary files! For whatever reason, the authors of MS-DOS chose to use the TEMP variable to control where these temporary files were created.
The pipe enables asynchronous IO. This can be important in case processes have two-way (full duplex) IO going on.
Okay Do I Care?
Yes, no, maybe.
You mostly don't. The ipstream/opstream classes are 100% compatible with std::istream/std::ostream, so if you had a function that expects them:
void simulate_input(std::ostream& os)
{
for (int i = 0; i < 10; ++i) {
os << "_ZN5boost7process8tutorialE" << std::endl;
}
}
You can perfectly use it in your sample:
bp::opstream in;
bp::ipstream out;
bp::child c("c++filt", bp::std_out > out, bp::std_in < in);
simulate_input(in);
in.close();
c.wait();
When You Definitely Need It
In full-duplex situations where you could easily induce a deadlock where both programs are waiting for input from the other end because they're doing the IO synchronously.
You can find examples + solution here:
How to reproduce deadlock hinted to by Boost process documentation?
boost::process::child will not exit after closing input stream
Boost::Process Pipe Streams and Unit Test
Suppose you declare an instance of std::ifstream or std::ofstream but is_open() returns 0
Example:
std::ifstream file("myfile.txt");
if (!file.is_open()) {
printf("Could not open file\n");
return;
}
Since the file never opened, do I still need to call file.close() after the printf statement?
No, you can only close an open file (similar to how you cannot close an already closed door - there is nothing to be done).
Extra note: Please do not combine the C I/O library (Xprintf family of functions) with the C++ I/O library (iostreams).
Consider using code like this:
std::ifstream file("myfile.txt");
if (!file.is_open()) {
std::cerr << "Could not open file\n";
return;
}
Edit (reasons not to use C IO API and C++ IO API together):
Using both APIs imposes synchronization between them, with priority towards the C API (i.e. the C api remains as fast, but IO streams will become slower, due to synchronization requirements).
It is inconsistent, using two very different concepts/abstraction levels for the same task. In more complex code, you will have to write twice the error handling (they impose different styles of error handling in client code), have both their limitations and combine their bad aspects (C API is prone to buffer overflows/security issues on reading and fails silently, unless you pay lots of attention to writing/maintenance of each API call, C++ API calls can be verbose).
They do not need to be particularly close (or far) appart, it is simply a bad programming practice.
This is similar to using a generic ODBC C API for reading table1 of your database, and ActiveX Data Objects for reading table2 in the same database, and the same program, or using Qt for your development, then hacking over it with raw WinAPI calls: you have twice the problems to solve (and many times end up implementing the solution to the same problems twice).
I would like my program to read the standard output produced by another application. I am aware that I can use popen to do that and use fread to read that output.
Do you know whether is possible to use read (and possibly open)? I am working in LINUX with C/C++
You can get a file descriptor for read() by calling int fd = fileno(fp) for the FILE *fp you have got from popen(). But be aware that you must not mix calling read() and fread()!
EDIT
If you want to avoid popen(), you have to use pipe(), fork(), exec..() and dup2() like it's done here
I want to run another program from my C++ code. system() returns int, as every program can only return int to the os. However, the other program I want to call will generate a string that I need in my base app. How can I send it to the parent process?
The two apps will be in the same folder, so I think that the child app can save the string to "temp.txt" and then the main app may read and delete it (it's not performance critical process, I will call another process just to call open file dialog in my main opengl app). However this is a bit ugly solution, are there better cross platform solutions?
Thanks
You could use popen(), this opens a process where you can write and read data. AFIK this is also cross plattform
// crt_popen.c
/* This program uses _popen and _pclose to receive a
* stream of text from a system process.
*/
#include <stdio.h>
#include <stdlib.h>
int main(void) {
char psBuffer[128];
FILE *pPipe;
/* Run DIR so that it writes its output to a pipe. Open this
* pipe with read text attribute so that we can read it
* like a text file.
*/
if((pPipe = _popen("dir *.c /on /p", "rt")) == NULL)
exit(1);
/* Read pipe until end of file. */
while(!feof(pPipe)) {
if(fgets(psBuffer, 128, pPipe) != NULL)
printf(psBuffer);
}
/* Close pipe and print return value of pPipe. */
printf("\nProcess returned %d\n", _pclose(pPipe));
return 0;
}
Although it's not part of the C++ standard, nearly all reasonably current systems provide a popen (or _popen, in Microsoft's case) that will let you spawn a child process and read from its standard output as a C-style FILE * in the parent. At least if memory serves, popen is included in POSIX, so you can expect it to be present in essentially all Unix-like systems (and, as implied above, it's also available on Windows, at least with most compilers).
In other words, about the only place you'd likely encounter that it's not available would be something like a small embedded system where it might well be pretty meaningless (e.g., no file system to find the other executable in, and quite possibly no ability to create new processes either).
Though there is no standard way of achieving interprocess communication, there is a relatively pain free library, ported to many OS/compilers: Boost.Interprocess. It covers most necessities:
Shared memory.
Memory-mapped files.
Semaphores, mutexes, condition variables and upgradable mutex
types to place them in shared memory and memory mapped files.
Named versions of those synchronization objects, similar to
UNIX/Windows sem_open/CreateSemaphore API.
File locking.
Relative pointers.
Message queues.
I have a bunch of legacy code that uses C-style file I/O (using FILE* streams, fprintf, fscanf, etc.) that I'd like to update to operate off of other data sources (e.g., downloading from or posting to the web, or to or from a custom data transfer cable).
If it used iostreams, this would be an easy update, but it doesn't.
Are there any C or C++ libraries for I/O that support different kinds of streams but that are source-compatible or mostly source-compatible with C's FILE* stream functions?
If source compatibility is too much to ask, I'd settle for "able to be search-and-replaced with a Perl script." fprintf and fscanf seem to be the major obstacles here; I haven't found any iostream-compatible versions of these.
I'm looking for a solution that will work in both Windows and Linux.
Example
As a trivial example, I have code like the following:
FILE *input = fopen(filename, "rt");
fscanf(input, "%d,%d,%d,%f", &f_id, &f_units, &h_index, &h_default);
fclose(input);
I'd like to update it to handle data that's of the same format but may come from other sources, without having to worry about the intricacies of how exactly to replicate fscanf's semantics myself (since I have many, many fscanf and fprintf calls that would need updating). Something like this would work, except that I've been unable to find anything like istream_scanf:
std::istream *input = make_http_istream(url);
istream_scanf(*input, &f_id, &f_units, &h_index, &h_default);
delete input;