Is there a way to execute a callback function, or otherwise call a predefined function whenever data is written to a standard stream such as stderr or stdout? Ideally, this could be used to allow an application to output as normal with printf in the case of stdout or fprintf for other FILE streams, and conditionally perform additional tasks such as assert dependent on current settings. This could have the benefit of automatically triggering this error handling code when other libraries output to the stream.
I know that output to stderr and stdout can be redirected to other FILE handles using std::freopen. Is it practical to implement an alternate FILE stream which provides this behavior, or would that require re-implementing a great deal of standard library functions?
Standards-compliant C++ suggestions (including C++11) would be preferred, although I would be open to Windows only solutions if necessary.
I've since had an attempt at implementing a streambuffer as suggested by doomster, with a little help from Filtering Streambufs by James Kanze. Unless any other suggestions come up, it seems like this is the closest you can get to the original suggestion. It won't intercept C-style output, which I suspect that is either impossible or at impractical, but it otherwise provides all the functionality I wanted.
You can replace the streambuffer in order to intercept any output to those streams, see the rdbuf() function. I think you only need to implement the overflow() function in the streambuffer class.
Related
I have a console app written with C++ that needs to accept text input from another process via writeline (followed by a deadline.) I'm assuming that such needs to be done via STDIN, but it also needs to work in the fastest way possible. My console app then needs to reply to the process back.
I haven't done console programming for awhile now, but I remember from C classes at school that there's a lot of C-type functions, like fgets, getline, etc. but what I remember is that they seemed to be quite slow.
So is there any way to do this exchange (again, "quick in and then out") with WinAPIs?
The fastest method in theory will almost certainly be the system
level input routines, since both the stdin (in C, but also
available in C++) and std::cin build on these. On the other
hand, they have generally been optimized for the platform, so
unless you can find the optimal configuration yourself (e.g.
things like buffer size), you might not gain much, if anything:
calling read (Unix) or ReadFile (Windows) for each character
will probably be slower than using something like
std::getline.
The other question is what you plan on doing with the data after
you've read it. Functions like read or ReadLine give you
a buffer (char[]) with a certain number of characters; you
then have to analyse it, break it into lines, etc. Functions
like std::getline give you an std::string with the line in
it. If you're a really skilled C++ programmer, you can probably
organize things so that the actual data are never moved from the
char[], but this would require re-implementing a lot of things
that are already implemented in the standard library. The use
of templates in the standard library means that you don't have
to implement nearly as much as you would have to otherwise, but
you'll still need to create an equivalent to std::string which
maintains two iterators (char const*) rather than the data
itself.
In the end, I'd start by writing the application using
std::getline and std::string. Once it's working, I'd see
what its actual performance is, and only then, if necessary,
consider ways of improving it.
CreateFile vs fopen vs ofsteam - advantage & disadvantage?
I heard that CreateFile powerful but only for windows.
Can you tell what should I use ( on windows) and why?
It depends on what you're doing. For sequentially reading and writing text files, iostream is definitely the way to go. For anything involving transactional security or non-standard devices, you'll have to access the system directly (CreateFile or open). Even then, for sequential reading and writing of text, the best solution is to define your own streambuf, and use that with iostream.
I can't think of any context where fopen would be preferable.
Unless you need the features provided by the Windows file functions (e.g. overlapped I/O) then my suggestion is going with either iostreams in C++ or FILE (fopen and friends) in C.
Besides being more portable, you can also use formated input/output for text files, and for C++ it's easy to overload the output/input operators for your classes.
If you want to use Windows file memory mapping you should use CreateFile (e.g. the HANDLE passed to CreateFileMapping API is the return value of CreateFile). Moreover, CreateFile offers higher customization options than C and C++ standard file API.
However, if you want to write portable code, or if you don't need Windows-specific features, C and C++ standard file APIs are just fine.
In some tests, when processing large data, I noted some performance overhead of C++ I/O streams vs. raw C file API; if you happen to be in cases like this, you could simply wrap raw C file API in some C++ RAII class, and still use it in C++ code.
Unless you absoulutely need the extra functionality provided by OS API functions (like CreateFile) I'd reccommend using the standard library functions (like fopen or ofstream). This way your program will be more portable.
The only real advantage of using CreateFile that I can think of is overlapped I/O and maybe finer grained access rights.
In most cases you will be better of using fopen in C or ofstream in C++. CreateFile gives some extra control over sharing and caching but does not provide formatting functionality.
I copied my answer from
fopen or CreateFile in Windows
which was closed for some reason which escapes me...
There is no defined way for fopen() to return the system error code. There might be an undefined way to access errno, but this might or might not be identical with the system error code.
Also, I don't think that there is an defined way to access the real system handle (of type HANDLE) which in turn you might want to use to pass on to one of the many win64 system calls which expect such a system handle (e.g. memory mapped IO)
Using open() an integer represents the file handle, which is not the system handle (on windows).
fopen() does not throw an exception in case of error. In order to have some RAII you would need to wrap it into a class.
Wrapping CreateFile() into a class, is not more expensive than wrapping fopen() or open() into a class.
Using the C++ feature (std::ofstream, std::ifstream) to write/read to/from files suffers from the same problem as fopen():
It does not throw by default on error. In order to enable this feature you need to call some method instead of being able to use some constructor argument -- means for RAII you would need to derive this class (in order to use it as a member/base class which throws on error).
It is undefined if one is able to retrieve the system error code from the exception thrown or if the message returned from what() tells you anything about the system error.
Using this stream interface there is no real pluggable interface to define the source or destination of reading from or writing to. Overloading the stream interface is quite cumbersome and error prone.
Using C like programming (paying attention to or ignoring return codes and manually writing cleanup code) is the source of much evil (remember heart-bleed?)...
Conclusions:
write a resource wrapper for CreateFile()/CloseHandle(). Resource wrapper is a class, which performs the do-action in the constructor and the undo-action in the destructor and throws an exception in case of error. There are many such pairs of system calls in every OS but especially in Win64.
Write a system error exception class (to be used for the above class in case of CreateFile() fails and for all the other system errors) or investigate, what the new system_exception class (in C++0x) is actually doing and if it is sufficient.
write a functional wrapper for ReadFile() and WriteFile() which converts a system error into a system exception object thrown...
Potentially define your own interface to write to somewhere and read from somewhere so that you can implement other things independent from the type of source/destination to read from/write to.
Writing a cache class which allows you to cache reading from somewhere or writing to somewhere is also child play. Of course the cache class should not know nor care about the source/destination you're writing to/reading from.
Don't be scared about these small tasks. You will actually know, what is happening in your code and these little pieces of code should be negligible (in amount of lines of code) compared to the code calling it. Also if you're using RAII for everything, the code calling into these utility classes, will be considerable less compared to when not using RAII and having to use two- or even more-step initialization and considerable less error prone. Replacing these utility classes with equivalent utility classes for other OS is also child play (using open()/close()/read()/write() on UNIXs).
And for the sake of the previous millennia don't read the google programming guidelines!
I just learned of the existence of the ios_base::sync_with_stdio function, which basically allows you to turn off (or on if you already turned it off) the synchronization between iostream streams that are used in C++ and the cstdio streams that are part of Standard C.
Now, I always thought that stdout, stderr and stdin in C were essentially wrapped in a set of objects in C++ in the iostreams classes. But if they have to be synchronized with each other, this would indicate that C++'s iostream classes are not a wrapper around C's stdin etc.
I'm quite confused by this? Can someone clarify how C++'s iostream and C's stdio are different things that do exactly the same thing, just at a different level of abstraction? I thought they were the same thing!?
How it is that they have to be synchronized? I always thought they were the same thing, one wrapping the other, essentially.
The C and C++ standards make no requirements on how things are implemented, just on what the effect of certain operations is. For the <stdio> vs. <iostream> functionality this means that one could wrap the other, both could be essentially the same, or that they are either entirely independent. Technically, using a common implementation would be ideal for several reasons (e.g. there would be no need for explicit synchronization and there would be a defined mechanism to extend FILE* for user defined systems) but I'm not aware of any system which actually does this. Having one implementation be a wrapper of the other is possible and implementing <iostream>s in terms of <stdio> was a typical implementation choice although it has the drawback that it introduces an extra cost for certain operations and most C++ standard libraries have moved on to use entirely separate implementations.
Unfortunately, both the wrapped and the independent implementation share a common problem: I/O is hideously inefficient when done one character level. Thus, it is essentially mandatory to buffer characters and read from or write to a buffer. This works nicely for streams which are independent of each other. The catch are the standard C streams stdin, stdout, stderr and their C++ narrow character counterparts std::cin, std::cout, std::cerr/std::clog and C++ wide character counterparts std::wcin, std::wcout, std::wcerr/std::wclog, respectively: what happens when a user reads both from stdin and std::cin? If either of these stream read a buffer of characters from the underlying OS stream the reads would appear out of order. Similarly, if both stdout and std::cout used independent buffers characters would appear in unexpected order when a user writes both to both streams. As a result, there are special rules on the standard C++ stream objects (i.e. std::cin, std::cout, std::cerr, and std::clog and their wide character counterparts) which mandate that they synchronize with their respective <stdio> counterpart. Effectively, this means that specifically these C++ objects either use a common implementation directly or that they are implemented in terms of <stdio> and don't buffer any characters.
It was realized that the cost of this synchronization is quite substantial if the implementations don't share a common base and may be unnecessary for some users: if a user only uses <iostream> he doesn't want to pay for the extra indirection and, more importantly, he doesn't want to pay for the extra costs imposed by not using a buffer. For careful implementations the cost of not using a buffer can be quite substantial because it means that certain operations end up having to do a check and possibly a virtual function call in each iteration rather than only once in a while. Thus, std::sync_with_stdio() can be used to turn this synchronization off which may mean that the standard stream objects change their internal implementation more or less entirely. Since the stream buffers of the standard stream objects can be replaced by a user, unfortunately, the stream buffers can't be replaced but the internal implementation of the stream buffer can be changed.
In good implementations of the <iostream> library all this only affects the standard stream objects. That is, file streams should be entirely unaffected by this. However, if you want to use the standard stream objects and want to achieve good performance you clearly don't want to mix <stdio> and <iostream> and you want to turn synchronization off. Especially, when comparing I/O performance between <stdio> and <iostream> you should be aware of this.
Actually the stdout, stderr and stdin are the file handlers of OS. And FILE structure of C as well as iostream classes of C++ are both wrappers of those file handlers. Both iostream classes and FILE structure may have their own buffers or something else that needs to be synchronized between each other to make sure that input from file or output to the file is done correctly.
Okay, here's what I've found.
Actually, the I/O is ultimately performed by native system calls and functions.
Now, take Microsoft Windows for example. There are actually available handles for STDIN , STDIO etc (see here). So basically, both the C++ iostream and C stdio call native system functions, the C++ iostream does not wrap C's I/O functions (in modern implementations). It calls the native system methods directly.
Also, I found this:
Once stdin, stdout, and stderr are redirected, standard C functions such as printf() and gets() can be used, without change, to communicate with the Win32 console. But what about C++ I/O streams? Since cin, cout, cerr, and clog are closely tied to C’s stdin, stdout, and stderr, you would expect them to behave similarly. This is half right.
C++ I/O streams actually come in two flavors: template and non- template. The older non-template version of I/O streams is slowly being replaced by a newer template style of streams first defined by the Standard Template Library (STL) and which are now being absorbed into the ANSI C++ standard. Visual C++ v5 provides both types and allows you to choose between the two by including different header files. STL I/O streams work as you would expect, automatically using any newly redirected stdio handles. Non-template I/O streams, however, do not work as expected. To discover why, I looked at the source code, conveniently provided on the Visual C++ CD-ROM.
The problem is that the older I/O streams were designed to use UNIX-style "file descriptors," where integers are used instead of handles (0 for stdin, 1 for stdout, and so on). That’s convenient for UNIX implementations, but Win32 C compilers have to provide yet another I/O layer to represent that style of I/O, since Win32 does not provide a compatible set of functions. In any case, when you call _open_osfhandle() to associate a new Win32 handle with (for example) stdout, it has no effect on the other layer of I/O code. Hence, file descriptor 1 will continue using the same underlying Win32 handle as before, and sending output to cout will not produce the desired effect.
Fortunately, the designers of the original I/O stream package foresaw this problem and provided a clean and useful solution. The base class ios provides a static function, sync_with_stdio(), that causes the library to change its underlying file descriptors to reflect any changes in the standard I/O layer. Though this is not strictly necessary for STL I/O streams, it does no harm and lets me write code that works correctly with either the new or old form of I/O streams.
(source)
Hence calling sync_with_stdio() actually changes the underlying file descriptors. It was in fact added by the designers to ensure compatibility of the older C++ I/O with systems like Windows-32 which used handles instead of integers.
Note that Using sync_with_stdio() is not necessary with modern C++ template-based STL I/O.
They are the same thing, but they might also be buffered separarately. This could affect code that mixes the use of C and C++ I/O, like this
std::cout << "Hello ";
printf("%s", "world");
std::cout << "!\n";
For this to work, the underlying streams must be synchronized somehow. On some systems, this might mean that performance could suffer.
So, the standard allows you to call std::sync_with_stdio(false) to say that you don't care about code like this, but would prefer to have the standard streams work as fast as possible if it makes a difference. On many systems it doesn't make a difference.
One can be a wrapper around the other (and that works both ways. You could implement stdio functions by using iostream and vice versa. Or you can write them completely independently.
And sync_with_stdio guarantees that the two streams will be synchronized if it's enabled. But they can still synchronize when it's disabled too, if the really want to.
But even if one is a wrapper around the other, one might still have a buffer that the other doesn't share, for example, so that synchronization is still necessary.
I have code written in C, which I'm working towards making a library from which I can access functions from other languages.
I am presently writing a C++ client to this code, to figure out what changes are necessary to provide support for including my library into a C++ application.
Normally, the C-based binary analyzes some input, and it then outputs its data to stdout through a series of fprintf statements.
Is there a way to redirecting output from the relevant C functions sent to stdout?
I am not a C++ expert, but I think what I would like to do is provide a subclass of a C++ output stream buffer (ostream and ofstream?), which contains the same output that would otherwise get sent by the relevant C functions to stdout.
I would like the option of redirecting this data to a file stream or to standard output within the C++ application, if possible.
How might I do this? Can I leave my C code as-is and redirect stdout from within the C++ code?
EDIT
I can edit the C functions in question, so that a file pointer or descriptor is an argument. Would this help, being able to pass in where the output goes? What's unclear to me is how I would translate C++ IO and C IO. I can't pass in a C++ iostream to a C function, or can I?
In a comment I mentioned that C++ std::cout and C stdout are related in a tricky way, and that you can use stdout in C++ by including <cstdio>. I also gave the following link: http://gcc.gnu.org/onlinedocs/gcc-4.6.2/libstdc++/manual/manual/io_and_c.html
However, you might consider changing the API of your library, and provide functions which write to a user supplied stream, and/or to a user supplied file path.
On Linux, you might also use open_memstream or fopencookie
If I've understood your question correctly, no, at least with standard conforming code.
What could be related is std::ios_base::sync_with_stdio which ensures that IO made on some C++ streams are correctly synchronized with IO make of C streams. So stdout is synchronized with cout, stdin with cin and stderr width cerr. But that isn't designed to allow changing the streambuf in cout and have the change propagated to stdout (which is how I understood your request), and I'm far from sure on the intented effect if you do a freopen on stdout.
You need to set up a pipe, and run the C program as a stand-alone process with its stdout connected to the input end of the pipe. Your C++ program then needs to read from the output end of the pipe, and it will then get all the output from the C process.
UPDATE: I re-read the question a bit more after the downvotes and comments. I (now) realize the C code is running as a library, i.e. linked into the same executable as the C++ code.
I don't think you can forcibly redirect stdout for the C code without affecting the C++'s program's output stream at the same time, which seems like a very high price to pay in order to have a library.
I still think the solution is to "sandbox" the C code in a process of its own, with a redirected output that you can then read as desired. It would of course be better to re-write the C functionality to return results instead of printing them, but I assume that's not an option.
You say that the C code uses fprintf(), i.e. the variant with an explicit file handle, so perhaps you can use that and make it accept a FILE * from the outside, in which case you can feed it a pipe without having to subprocess it.
Very interested in seeing other answers, perhaps there's an approach I'm missing.
UPDATE 2: If you have the chance to control which FILE * you pass into the C library, then it's easy:
To have the output appear with the C++ program's ordinary output, just pass in stdout.
To capture the output to a file, fopen() the file and pass in the resulting FILE *.
If you want to do both at the same time, that's a bit more troublesome, again. In that case I would, again, recommend a pipe or socket so that the C++ side is responsible for reading out the output generated by the C library, and do what it wishes with it.
I implemented a small program that can extract (and via fuse mount) a certain archive format. I use boost::filesystem::ifstream, but on error (e.g. the file a user wants to extract does not exist) I get very nondescript error messages. I wonder is there a way to get better error messages for IO related problems in C++?
On a related note I wonder whether I should have used C's FILE* or in the case of the fuse filesystem just plain file descriptors? Because strerror(errno) is way better than what iostreams are giving me.
We couldn't find any better way than using boost::iostreams and implementing our own file-based sink and source.
If you want, you can grab the source code here (Apache-licensed):
http://sourceforge.net/projects/cgatools/files/1.3.0/cgatools-1.3.0.9-source.tar.gz/download
the relevant files are:
cgatools/util/Streams.[ch]pp
Since your using the filesystem library anyway, you could test to see if the file exists prior to trying to access it with a stream. This would avoid your bloat concerns, but it would not operate in the same sense as what you're looking for, i.e. the stream itself would not perform the existence check.
However, since you are using boost::filesystem::ifstream, I'm assuming that you are using that because you are using boost::filesystem::path. In boost's implementation of ifstream, they inherit from std::basic_ifstream and override two functions: the constructor and open. So, if you want better error reporting you could simply do they same thing, inherit from boost's implementation and override those two functions to provide the checking you wish. Additional bloat: probably not a lot, and it incorporates the behavior you wish into the stream itself.