Documentation states that FILE is object type that identifies a stream. So, is it possible to get the stream object associated with a FILE?
For example, I'd like to get std::cout object from stdout FILE pointer, or std::cerr from stderr etc. More generally I want to write a function that redirects a given stream and sets the custom streambuf to it, something like this:
void redirect(FILE* file, std::ios stream) {
freopen_s((FILE**)file, "CONOUT$", "w", file);
stream.rdbuf(customBuffer);
}
used to redirect streams
redirect(stdout, std::cout);
redirect(stderr, std::cerr);
It seems redundant to have 2 parameters, since both parameters are always associated with each other.
The C++ standard library includes the C standard library. A FILE is a C stream, which is quite a different animal than a C++ iostream. It is possible for an std::stream implementation to rely of an underlying FILE, but this is not required by the standard, and even in that case there is no way to retrieve it.
What is possible is to build a custom std::streambuf that explicitly uses an underlying FILE *, and use it in a std::stream. std::basic_streambuf is one of the few classes from the C++ standard library that is explicitely designed as a base class for custom derivation. Unfortunately I could not find a tutorial for it, but the class contains a number of virtual methods that you just have to override. It is not exactly an easy path, but is possible with some works, heavy testing, and eventually some help from SO if you get stuck somewhere. But a full implementation is far beyond a SO answer.
TL/DR: there is no underlying std::stream associated with a FILE but with some work you can build a custom stream_buffer that will use an underlying FILE *. Though those are rather advanced operations...
While it is not possible to cleanly do this in C++ you could do something like this.
FILE * file = popen("someFile")
const unsigned BUFF = 2048;
string total;
bool done = false;
while (!done) {
vector<char> cBuf[BUFF];
size_t read = fread((void *)&cBuf[0], 1, BUFF, f);
if (read)
{
total.append(cBuf.begin(), cBuf.end());
}
if (read < BUFF)
{
done = true;
}
}
pclose(f);
istringstream filey(total);
Hope this helps.
Related
Is there a single standard C/C++ function which takes either a file handle/pointer or a pointer to an in-memory buffer and reads data from the file/buffer?
I have a function which pulls data from a file, performs operations on said data, and sends it out over a socket. I also have a function that takes data in the form of a char buffer, performs exactly the same operations on that data, and sends it over a socket. This problem isn't that hard. I just thought it would be convenient if there was a function like
read(void *dest, void *src, int src_type, size_t amount)
In C++ you can use std::istream as an abstraction on top of a file with an std::ifstream implementation, or an in-memory buffer with an std::istringstream implementation.
The beauty of this approach is that your function does not need to know about the implementation type, because receiving the argument by reference would be sufficient:
void readData(std::istream& in_data) {
...
}
The caller would then construct an appropriate implementation, and pass it to readData.
Sounds like you want to use the operator>>() on an input stream (std::istream), which can either be a filestream, a iostream or a stringstream
I don't know about windows but it will work in Linux.
void readif(int fd ,char *src,size_t nbytes){
read(fd,src,nbytes);
//error handler code
//Other stuffs
}
int fd = open(file);
//for std::input fd = 0
//for in memory data use mmap
mmap
open
read
I've created an fstream object to write info to files.
I write strings to the new file like
fStreamObject << "New message.\n";
because I want each << to print a string to the next line.
I want to be able to set a property and make a call like
fstreamObject << "New message.";
which will write the string to the next line.
Are there flags/settings for fstream objects that allows this to be done?
I've seen the different file modes (i.e. ofstream::in, ofstream::out, etc.), but I couldn't find one that auto writes to a new line. Also, I'm not looking to write my own solution. I want to be able to use a built in feature.
No, there are no readily configurable capabilities of that sort within the standard streams.
You may have to subclass the stream type and fiddle with operator<< to get this to work the way you want, or do it with a helper function of some description:
fstreamObject << nl("New message.");
(but that's hardly easier than just having the \n in there (for a string, anyway).
It depends on what you mean by "setting the stream". If we consider this to be fairly broad then the answer happens to be "yes"!
Here is how:
Create a stream buffer which inserts a newline every time it is flushed, i.e., when sync() is called. Otherwise it just forwards characters.
Change the file stream's stream buffer to use this stream buffer filtering to the file stream's stream buffer.
Set the flag std::ios_base::unitbuf which causes a flush after every [properly written] output operation.
Here are is the example code to do just that:
#include <iostream>
class newlinebuf
: public std::streambuf {
std::ostream* stream;
std::streambuf* sbuf;
int overflow(int c) { return this->sbuf->sputc(c); }
int sync() {
return (this->sbuf->sputc('\n') == std::char_traits::eof()
|| this->sbuf->pubsync() == -1)? -1: 0;
}
public:
newlinebuf(std::ostream& stream)
: stream(&stream)
, sbuf(stream.rdbuf(this)) {
stream << std::unitbuf;
}
~newlinebuf() { this->stream->rdbuf(this->sbuf); }
};
int main() {
newlinebuf sbuf(std::cout);
std::cout << "hello" << "world";
}
Although this approach work, I would recommend against using it! On problem is that all composite output operators, i.e., those using multiple output operators to do their work, will cause multiple newlines. I'm not aware of anything which can be done to prevent this behavior. There isn't anything in the standard library which enables just configuring the stream to do this: you'll need to insert the newline somehow.
No, the C++ streams do not allow that.
There is no way to decide where one insertion stops and the next starts.
For example for custom types, their stream-inserters are often implemented as calls to other stream-inserters and member-functions.
The only things you can do, is write your own class, which delegates to a stream of your choosing, and does that.
That's of strictly limited utiliy though.
struct alwaysenter {
std::ostream& o;
template<class X> alwaysenter& operator<<(X&& x) {
o<<std::forward<X>(x);
return *this;
}
};
I have some data that I want to format and output, either in raw text file or .gz compressed text file.
Thus, I wanted to do something like this :
shared_ptr<ofstream> file = make_shared<ofstream>(filename.c_str());
shared_ptr<ostream> stream;
if (compressed) {
stream = getCompressedStream(*file);
} else {
stream = std::move(file);
}
for (auto c = data.begin(); c != data.end(); ++c) {
*stream << **c << "\t" << (*c)->getThing() << endl;
}
With getCompressedStream a function that decrypt the stream and return a new non encrypted stream :
std::unique_ptr<std::ostream> getCompressedStream(std::ofstream& file)
{
typedef io::filtering_ostream filter;
std::unique_ptr<filter> out = std::unique_ptr<filter> (new filter());
out->push(io::gzip_compressor());
out->push(file);
return std::move(out);
}
So I wanted getCompressedStream to abstract calls to boost lib, so that I only use std streams in my main program.
It's not working : the .gz file is damaged/not readable.
According to this thread, the output of a filtering_stream is done in the destruction of the object. Thus, I don't know if it can be done cleanly with shared_ptr. I don't know whether file or stream is destroyed first, and I guess this leads to issues.
Do you think implementing getCompressedStream this way is possible? What would you change?
Thanks
Edit : It works if I switch to regular pointers instead of shared pointers, and delete the stream explicitely before the file.
I'm not sure. The real problem is that you never explicitly
close the ofstream, so you have no idea as to what is going
on. (Except in the case of exceptions, where you're going to
delete the new file anyway, you must check the status of the
ofstream object after the close, to know whether everything
succeeded or not.) More generally, your solution seems overly
complicated to me; you're trying to do too much in a single
function. The way I usually do this (with my own filtering
streambuf, which predate the boost versions by some years) is to
do something like:
std::ofstream file( filename.c_str() );
if ( compressed ) {
CompressedStreambuf c( file );
outputData( file );
} else {
outputDate( file );
}
file.close();
if ( !file ) {
remove( filename.c_str() ); // Since it isn't complete
// Ensure that program return code is failure.
}
Of course, my own filtering streambuf work slightly differently
than those of boost: the constructor taking an ostream inserts
the filter in front of the ostream, and the destructor flushes
and removes the filter (a variant of RAII). But it shouldn't be
too difficult to make a wrapper of the Boost classes to support
this.
The advantage of this solution is that everything is guaranteed
to be constructed and destructed in the correct, nested order,
because there is no dynamic allocation which would allow an
arbitrary order.
Is there an easy way to check if something was serialized in stl::ostream. I am looking for something like:
some preparation
// ... a very complex code that may result in adding or not to the stream,
// that I will prefer not to change
check if the stream has something added
Note that this will need to works recursively. Is using register_callback is a good idea or there is easier way?
First the immediate question: register_callback() is intended to deal with appropriate copying and releasing of resources stored in pword() and will have operations only related to that (i.e., copying, assigning, and releasing plus observing std::locale changes). So, no, that won't help you at all.
What you can do, however, is to create a filtering stream buffer which observes if there was a write to the stream, e.g., something like this:
class changedbuf: std::streambuf {
std::streambuf* d_sbuf;
bool d_changed;
int_type overflow(int_type c) {
if (!traits_type::eq_int_type(c, traits_type::eof())) {
this->d_changed = true;
}
return this->d_sbuf->sputc(c);
}
public:
changedbuf(std::streambuf* sbuf): d_sbuf(d_sbuf), d_changed() {}
bool changed() const { return this->d_changed; }
}
You can use this in place of the std::ostream you already have, e.g.:
void f(std::ostream& out) {
changedbuf changedbuf(out.rdbuf());
std::ostream changedout(&changedbuf);
// use changedout instead of out; if you need to use a global objects, you'd
// replace/restore the used stream buffer using the version of rdbuf() taking
// an argument
if (changedbuf.change()) {
std::cout << "there was a change\n";
}
}
A real implementation would actually provide a buffer and deal with proper flushing (i.e., override sync()) and sequence output (i.e., override xsputn()). However, the above version is sufficient as a proof-of-concept.
Others are likely to suggest the use of std::ostringstream. Depending on the amount of data written, this can easily become a performance hog, especially compared to an advanced version of changedbuf which appropriately deals with buffering.
Are you passing the stream into the complex code, or is it globally visible? Can it be any kind of ostream or can you constrain the type to ofstream or ostringstream?
You may be able to use tellp to determine whether the file position has changed since your preparation code, if your ostream type supports it (such as with most fstreams). Or, if you're passing the stream in, you could pass an empty ostringstream in and check that it's not empty when the string is extracted to be printed out.
It's not entirely obvious which solution, if any, would be appropriate for you without knowing more about the context of your code and the specifics of your problem. The best answer may be to return (or set as a by-reference out-parameter) a flag indicating whether the stream was inserted into.
In my application, I want to redirect the output that would normally go to the stdout stream to a function I define. I read that you can redirect stdio to a file, so why not to a function?
For example:
void MyHandler( const char* data );
//<<Magical redirection code>>
printf( "test" );
std::cout << "test" << std::endl;
//MyHandler should have been called with "test" twice, at this point
How can I achieve this / similar behaviour?
#Konrad Rudolph is right, you can totally do this, easily, at least for cout/cerr/clog. You don't even need your own streambuf implementation, just use an ostringstream.
// Redirect cout.
streambuf* oldCoutStreamBuf = cout.rdbuf();
ostringstream strCout;
cout.rdbuf( strCout.rdbuf() );
// This goes to the string stream.
cout << "Hello, World!" << endl;
// Restore old cout.
cout.rdbuf( oldCoutStreamBuf );
// Will output our Hello World! from above.
cout << strCout.str();
Same thing works for cerr and clog, but in my experience that will NOT work for stdout/stderr in general, so printf won't output there. cout goes to stdout, but redirecting cout will not redirect all stdout. At least, that was my experience.
If the amount of data is expected to be small, the freopen/setbuf thing works fine. I ended up doing the fancier dup/dup2 thing redirecting to a pipe.
Update: I wrote a blog post showing the dup2 method I ended up using, which you can read here. It's written for OS X, but might work in other Unix flavors. I seriously doubt it would work in Windows. Cocoa version of the same thing here.
Invoking a callback function whenever something’s written to the underlying standard output stream is hard: it requires overriding low-level, system-specific functions (on POSIX systems, it would require at least overriding write, and the way this is called might depend on the standard library implementation, so it might be different between e.g. glibc and musl).
But depending on what exactly you’re after you can solve this in C++ without resorting to low-level OS specific functions, by manipulating the C++ stream buffers directly.
For this you need to create your own std::streambuf implementation, i.e. your own stream buffer.
Once you have that, you can redirect the std::cout stream by switching the buffer:
auto buf = callback_streambuf(MyHandler);
auto pold_buffer = std::cout.rdbuf(&buf);
std::cout << "Hello" << std::cout;
// Restore original buffer:
std::cout.rdbuf(pold_buffer);
However, what this implementation won’t do is call your callback function exactly twice. Instead, the number of calls will depend on several factors, but it generally will not depend on the number of stream insertions (<<), and there is no way around that!
For the specific example above, the callback is called once, with the data "Hello\n".
The following minimal implementation illustrates how to get a streambuf to call your handler:
class callback_streambuf : public std::streambuf {
public:
callback_streambuf(std::function<void(char const*, std::streamsize)> callback) : callback(callback) {}
protected:
std::streamsize xsputn(char_type const* s, std::streamsize count) {
callback(s, count);
return count;
}
private:
std::function<void(char const*, std::streamsize)> callback;
};
This implementation has several caveats. For instance, it does the wrong thing when attempts are made to use it as an input stream. It doesn’t override overflow (since I don’t think this is ever called, although I find conflicting information about this on the internet; at any rate, adding overflow would be trivial). I didn’t implement synchronisation, so the callback will be called concurrently from multiple threads. Furthermore, there is no error handling, since the callback does not return a success status. I also had to change the signature of the callback to
void MyHandler(char const* data, std::streamsize count);
The second parameter is required since data isn’t a string, it’s a raw char buffer, so there is no way to determine its length intrinsically, and MyHandler can do nothing useful with the data without knowing its length.
Answer: Yes you can, via a dup. freopen will only reopen stdout to a file, as you talked about.
Check out How to buffer stdout in memory and write it from a dedicated thread
It's possible to disable stdin/stdout by dereferencing its pointer:
FILE fp_old = *stdout; // preserve the original stdout
*stdout = *fopen("/dev/null","w"); // redirect stdout to null
HObject m_ObjPOS = NewLibraryObject(); // call some library which prints unwanted stdout
*stdout=fp_old; // restore stdout
The std::cout object has a fixed meaning, and that is to output to the standard out stream. The user of your program gets to control where standard out is connected to, not you. What you can do is decide whether you wish to write to a file, to standard out or to any other output stream. So in your code you switch what stream you write to.
Again, the point of writing to the standard out stream is to give the user a flexibility in choosing where the output goes to. You're not supposed to redirect standard out; this is something the user is supposed to have the freedom to do.
Another thing is that you shouldn't mix C IO and C++ IO in a C++ program. Choose which IO library you wish to work with and stick to it.
That said, you can in C++ quite elegantly switch streams for a function to take input from by templating the handler function on the template parameters of std::basic_istream<>. Then the function will read its input from the input stream independently of the real kind of stream it's working with. Here's an example:
#include<iostream>
#include<fstream>
#include<string>
template<class Ch, class Tr>
void dodge_this(std::basic_istream<Ch, Tr>& in)
{
// in is an input stream. read from it as you read from std::cin.
}
int main(int argc, char* argv[])
{
if( std::string(argv[1]) == "cin" ) {
dodge_this(std::cin);
} else if( std::string(argv[1]) == "file" ) {
std::ifstream file("input.txt");
dodge_this(file);
} else {
dodge_this(dev_null_stream); // i just made that up. you get the idea.
}
}
Another option is to place your handler class calls into the inherited streambuf class. I had a requirement to redirect cout to a Win GUI edit control in a dialog box that may be of some use. Here is the class code:
//-------------------------------- DlgStringbuf Definition -----------------------
class DlgStringbuf : public std::stringbuf
{
public:
DlgStringbuf(void) : _hwndDlg(NULL), _editControlID(0), _accum(""), _lineNum(0) {}
void SetDlg(HWND dlg, int editControlID)
{ _hwndDlg = dlg; _editControlID = editControlID; }
void Clear(void)
{ _accum.clear(); _lineNum = 0; }
protected:
virtual std::streamsize xsputn(const char* s, std::streamsize num)
{
std::mutex m;
std::lock_guard<std::mutex> lg(m);
// Prepend with the line number
std::string str(s, (const uint32_t)num);
str = std::to_string(_lineNum) + ": " + str + "\r\n";
// Accumulate the latest text to the front
_accum = str + _accum;
// Write to the Win32 dialog edit control.
if(_hwndDlg != NULL)
SetDlgItemTextW(_hwndDlg, _editControlID, (LPCWSTR)(std::wstring(_accum.begin(), _accum.end())).c_str());
_lineNum++;
return(num);
}//end xsputn.
private:
std::string _accum;
HWND _hwndDlg;
int _editControlID;
uint32_t _lineNum;
};//end DlgStringbuf.
//-------------------------------- DlgStream Definition ------------------------------
class DlgStream : public std::ostream
{
public:
DlgStream(void) : std::ostream(&_sbuff) {}
void SetDlg(HWND dlg, int editControlID)
{ _sbuff.SetDlg(dlg, editControlID); }
void Clear(void)
{ _sbuff.Clear(); }
private:
DlgStringbuf _sbuff;
};
...and in the WinMain, someplace after the dialog box and its edit control is created:
// Redirect all cout usage to the activity dlg box.
// Save output buffer of the stream - use unique pointer with deleter that ensures to restore
// the original output buffer at the end of the program.
auto del = [&](streambuf* p) { cout.rdbuf(p); };
unique_ptr<streambuf, decltype(del)> origBuffer(cout.rdbuf(), del);
// Redirect the output to the dlg stream.
_dlgStream.SetDlg(hwndActivityDlg, IDC_EDIT_ACTIVITY);
_dlgStream.copyfmt(cout);
cout.rdbuf(_dlgStream.rdbuf());
cout << "this is from cout";
You can use sprintf to write to a character array and then read the value:
char buf[1024];
sprintf(buf, "test");
MyHandler(buf);
there are also snprintf and a few others depending on platform