I am writing a file splitting program, to assist with using large files with iPod notes. I want to use tmpfile() in cstdio but it returns a file* not an fstream object. I know it's not possible in standard C++ but does anyone know any libraries that work well with the standard that have the ability to convert a FILE* to an std::fstream object? Or, if not is tmpfile() functionality available in the standard, or another library?
Thanks!
My OS is Windows XP and I use either Dev-C++ 4.9.9.2 or MS Visual Studio 2008 as my compiler.
If all you want is a temporary file, use tmpnam() instead. That returns char* name that can be used for a temporary file, so just open a fstream object with that name.
Something like:
#include <cstdio>
#include <fstream>
...
char name[L_tmpnam];
tmpnam(name);
//also could be:
//char *name;
//name = tmpnam(NULL);
std::fstream file(name);
You do have to delete the file yourself, though, using remove() or some other method.
You can use the benefits of c++ streams by pumping your data via the << syntax into a std::stringstream
and later write it the .str().c_str() you get from it via the the C-API to the FILE*.
#include <sstream>
#include <cstdio>
#include <string>
using namespace std;
int main()
{
stringstream ss;
ss << "log start" << endl;
// ... more logging
FILE* f_log = fopen("bar.log", "w");
string logStr = ss.str();
fwrite(logStr.c_str(), sizeof(char), logStr.size(), f_log);
fclose(f_log);
return 0;
}
Even if you manage to convert a FILE* to an std::fstream, that won't work as advertised. The FILE object returned by tmpfile() has a special property that, when close()'d (or when the program terminates), the file is automatically removed from the filesystem. I don't know how to replicate the same behavior with std::fstream.
You could use tmpnam mktmp to obtain a temporary file name, open it with a stream and then delete it with remove.
char *name;
ifstream stream;
name = mktmp("filename");
stream.open(name);
//do stuff with stream here.
remove(name);//delete file.
Instead of using std::fstream, you could write a simple wrapper class around FILE*, which closes it on destruction. Should be quite easy. Define operators like << as necessary.
Be sure to disallow copying, to avoid multiple close() calls.
g++ has __gnu_cxx::stdio_filebuf and __gnu_cxx::stdio_sync_filebuf, in ext/stdio_filebuf.h and ext/stdio_sync_filebuf.h. It should be straight-forward to extract them from libstdc++ if your compiler is not g++.
Related
I'm basically looking for a C++ version of fdopen(). I did a bit of research on this and it is one of those things that seems like it should be easy, but turns out to be very complicated. Am I missing something in this belief (i.e. it really is easy)? If not, is there a good library out there somewhere to handle this?
EDIT: Moved my example solution to a separate answer.
From the answer given by Éric Malenfant:
AFAIK, there is no way to do this in
standard C++. Depending on your
platform, your implementation of the
standard library may offer (as a
nonstandard extension) a fstream
constructor taking a file descriptor
as input. (This is the case for
libstdc++, IIRC) or a FILE*.
Based on above observations and my research below there's working code in two variants; one for libstdc++ and another one for Microsoft Visual C++.
libstdc++
There's non-standard __gnu_cxx::stdio_filebuf class template which inherits std::basic_streambuf and has the following constructor
stdio_filebuf (int __fd, std::ios_base::openmode __mode, size_t __size=static_cast< size_t >(BUFSIZ))
with description This constructor associates a file stream buffer with an open POSIX file descriptor.
We create it passing POSIX handle (line 1) and then we pass it to istream's constructor as basic_streambuf (line 2):
#include <ext/stdio_filebuf.h>
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main()
{
ofstream ofs("test.txt");
ofs << "Writing to a basic_ofstream object..." << endl;
ofs.close();
int posix_handle = fileno(::fopen("test.txt", "r"));
__gnu_cxx::stdio_filebuf<char> filebuf(posix_handle, std::ios::in); // 1
istream is(&filebuf); // 2
string line;
getline(is, line);
cout << "line: " << line << std::endl;
return 0;
}
Microsoft Visual C++
There used to be non-standard version of ifstream's constructor taking POSIX file descriptor but it's missing both from current docs and from code. There is another non-standard version of ifstream's constructor taking FILE*
explicit basic_ifstream(_Filet *_File)
: _Mybase(&_Filebuffer),
_Filebuffer(_File)
{ // construct with specified C stream
}
and it's not documented (I couldn't even find any old documentation where it would be present). We call it (line 1) with the parameter being the result of calling _fdopen to get C stream FILE* from POSIX file handle.
#include <cstdio>
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main()
{
ofstream ofs("test.txt");
ofs << "Writing to a basic_ofstream object..." << endl;
ofs.close();
int posix_handle = ::_fileno(::fopen("test.txt", "r"));
ifstream ifs(::_fdopen(posix_handle, "r")); // 1
string line;
getline(ifs, line);
ifs.close();
cout << "line: " << line << endl;
return 0;
}
AFAIK, there is no way to do this in standard C++. Depending on your platform, your implementation of the standard library may offer (as a nonstandard extension) a fstream constructor taking a file descriptor (This is the case for libstdc++, IIRC) or a FILE* as an input.
Another alternative would be to use a boost::iostreams::file_descriptor device, which you could wrap in a boost::iostreams::stream if you want to have an std::stream interface to it.
There's a good chance your compiler offers a FILE-based fstream constructor, even though it's non-standard. For example:
FILE* f = fdopen(my_fd, "a");
std::fstream fstr(f);
fstr << "Greetings\n";
But as far as I know, there's no portable way to do this.
Part of the original (unstated) motivation of this question is to have the ability to pass data either between programs or between two parts of a test program using a safely created temporary file, but tmpnam() throws a warning in gcc, so I wanted to use mkstemp() instead. Here is a test program that I wrote based on the answer given by Éric Malenfant but using mkstemp() instead of fdopen(); this works on my Ubuntu system with Boost libraries installed:
#include <stdlib.h>
#include <string.h>
#include <assert.h>
#include <string>
#include <iostream>
#include <boost/filesystem.hpp>
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/iostreams/stream.hpp>
using boost::iostreams::stream;
using boost::iostreams::file_descriptor_sink;
using boost::filesystem::path;
using boost::filesystem::exists;
using boost::filesystem::status;
using boost::filesystem::remove;
int main(int argc, const char *argv[]) {
char tmpTemplate[13];
strncpy(tmpTemplate, "/tmp/XXXXXX", 13);
stream<file_descriptor_sink> tmp(mkstemp(tmpTemplate));
assert(tmp.is_open());
tmp << "Hello mkstemp!" << std::endl;
tmp.close();
path tmpPath(tmpTemplate);
if (exists(status(tmpPath))) {
std::cout << "Output is in " << tmpPath.file_string() << std::endl;
std::string cmd("cat ");
cmd += tmpPath.file_string();
system(cmd.c_str());
std::cout << "Removing " << tmpPath.file_string() << std::endl;
remove(tmpPath);
}
}
It actually is quite easy. Nicolai M. Josuttis has released fdstream in conjunction with his book The C++ Standard Library - A Tutorial and Reference. You can find the 184 line implementation here.
I've tried the solution proposed above for libstdc++ by Piotr Dobrogost, and found that it had a painful flaw: Due to the lack of a proper move constructor for istream, it's very difficult to get the newly constructed istream object out of the creating function. Another issue with it is that it leaks a FILE object (even thought not the underlying posix file descriptor). Here's an alternative solution that avoids these issues:
#include <fstream>
#include <string>
#include <ext/stdio_filebuf.h>
#include <type_traits>
bool OpenFileForSequentialInput(ifstream& ifs, const string& fname)
{
ifs.open(fname.c_str(), ios::in);
if (! ifs.is_open()) {
return false;
}
using FilebufType = __gnu_cxx::stdio_filebuf<std::ifstream::char_type>;
static_assert( std::is_base_of<ifstream::__filebuf_type, FilebufType>::value &&
(sizeof(FilebufType) == sizeof(ifstream::__filebuf_type)),
"The filebuf type appears to have extra data members, the cast might be unsafe");
const int fd = static_cast<FilebufType*>(ifs.rdbuf())->fd();
assert(fd >= 0);
if (0 != posix_fadvise(fd, 0, 0, POSIX_FADV_SEQUENTIAL)) {
ifs.close();
return false;
}
return true;
}
The call to posix_fadvise() demonstrates a potential use. Also note that the example uses static_assert and using which are C++ 11, other than that it should build just fine in C++ 03 mode.
Another non-portable solution is to use mmap (or its Windows' analogue) and then construct std::iostream from a pointer that mmap gave like so.
Yeah, it does not construct exactly an std::fstream, but this requirement rarely needs to be met because every piece of code should depend on stream interfaces (e.g. std::istream) rather than on their implementations.
I think this solution is more portable than use of STL implementation-specific hacks, because this way you only depend on an operating system, rather than on a specific implementation of STL for the same OS.
My understanding is that there is no association with FILE pointers or file descriptors in the C++ iostream object model in order to keep code portable.
That said, I saw several places refer to the mds-utils or boost to help bridge that gap.
I need to use an existing C library in a C++ project.
My problem is the C existing library uses FILE* as streams, is there a standard compliant way to use FILE* streams to put or to get from C++ streams?
I tried seeking FILE* in the C++11 standard, but it appears only if few examples.
-
EDIT, example.
We have a function which write in a FILE*:
fputs(char const*, FILE*)
We have an output C++ stream:
auto strm = std::cout;
Can I, in a standard compliant way, to use fputs to write to strm?
You can also think a similar input example with a C function that reads from a FILE* and a input C++ stream.
Check this answer ,
https://stackoverflow.com/a/5253726/1807864
#include <cstdio>
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main()
{
ofstream ofs("test.txt");
ofs << "Writing to a basic_ofstream object..." << endl;
ofs.close();
int posix_handle = ::_fileno(::fopen("test.txt", "r"));
ifstream ifs(::_fdopen(posix_handle, "r")); // 1
string line;
getline(ifs, line);
ifs.close();
cout << "line: " << line << endl;
return 0;
}
The C++ standard, with very few exceptions says that C functionality is all included. Certainly all the stadnard C FILE * functionality is supported [obviously subject to general support for FILE * on the platform - an embedded system that doesn't have a 'files' (e.g. there is no storage) in general will hardly have much useful support for FILE * - nor will C++ style fstream work].
So, as long as you don't try to mix reading/writing to the same file from both C++ fstream and C FILE * at the same time, it should work just fine. You will need to close the file between C++ and C access, however.
As far as I understand, doing so is non-portable and non-standard. Whether it works in practice or not is another question. I would create wrapper classes around your legacy code, or use FILE* streams directly like in C - they are not that bad. :)
Ok so I haven't used C++ since highschool (used to work in borland C++)
Now I want to solve a problem in C++, yet I don't understand why fstream doesn't work
For some reason ios::in doesn't work.
#include <fstream>
fstream f("Cities.txt,ios::in);
How do I use Fstream properly?
Thanks in advance!
Note : I'm using Visual Studio 2008
change from
fstream f("Cities.txt,ios::in);
to
std::fstream f("Cities.txt" , std::ios::in);
^^^ ^ ^^^
namespace you miss" namespace
done!
What you have learned in your highschool probably was way before C++ was standardized in '97. As per the standard, all C++ library functions are part of the std namespace. In order to use fstream which is part of the standard namespace, you have to qualify it with std:: so, that makes your syntax as
#include <fstream>
std::fstream f("Cities.txt",std::ios::in);
As an alternative to std::fstream, consider std::ifstream (and std::ofstream):
#include <fstream>
…
std::ifstream f("Cities.txt");
std::ofstream o("output.txt");
std::string s;
while( f >> s )
o << s;
Personally, I find this more convenient than specifying the open mode.
You have to first create an object of ifstream class and then open the file.
Do it this way.
#include <fstream>
std :: ifstream f ("Cities.txt",ios::in) ;
Then check whether it is open and start working with it.
You are also missing the " after file name.
You can also write
#include <fstream>
using namespace std;
fstream f("Cities.txt",ios::in);
The using directive allows you to not write std:: before everything. Beware, it might be bad practice, but in small programs it should not be an issue.
I've been looking for a good, portable way to create a zip file in C++ and been coming up short. I've read in various places that its' possible to use the Boost IOstream library, but I can't find any source code or even documentation on it in the reference:
http://www.boost.org/doc/libs/1_48_0/libs/iostreams/doc/index.html
Does anybody have a good reference? I've done a whole lot of Googling and not come up with much.
I do not think boost::iostreams can open a zip file. See Unziping a zip file with boost and Visual C++ 2005.
boost::iostreams can be used to compress streams or single files using zlib, gzip or bzip2. You may find some examples here:
zlib
gzip
bzip2
However, it can not read the directory information inside a zip file.
On the other hand, you need to compile boost using third party libraries: zlib and bzip2. See the installation information.
I wrote a simple zip file maker that allows use with iostreams.
It's included in the partio library
https://github.com/wdas/partio/blob/master/src/lib/io/ZIP.h
https://github.com/wdas/partio/blob/master/src/lib/io/ZIP.cpp
For example you can create a zip file with two files by doing
ZipFileWriter zip("foo.zip");
std::ostream* o = zip.Add_File("test.txt");
*o << "look a zip file" << std::endl;
delete o;
std::ostream* o2 = zip.Add_File("test2.txt");
*o2 << "look another file" << std::endl;
delete o2;
And then could read a file by doing
ZipFileReader zip("foo.zip");
std::istream* i = zip.Get_File("test.txt");
std::string foo;
*i >> foo;
std::cout << foo << std::endl;
delete i;
Maybe it's not a zip file, but rather a compressed file, if it otherwise can help with your intention.
Actually, I tested this:
#include <fstream>
#include <sstream>
#include <iostream>
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filter/zlib.hpp>
TEST(MyTests, SaveZipFile)
{
using namespace std;
ofstream file("a.z", ios_base::out | ios_base::binary);
boost::iostreams::filtering_streambuf<boost::iostreams::output> out;
out.push(boost::iostreams::zlib_compressor());
out.push(file);
stringstream sstr{"nihao"};
boost::iostreams::copy(sstr, out);
}
TEST(MyTests, OpenZipFile)
{
using namespace std;
ifstream file("a.z", ios_base::in | ios_base::binary);
boost::iostreams::filtering_streambuf<boost::iostreams::input> in;
in.push(boost::iostreams::zlib_decompressor());
in.push(file);
stringstream sstr;
boost::iostreams::copy(in, sstr);
cout << sstr.str() << endl;
ASSERT_EQ(sstr.str(), string("nihao"));
}
This is a useful streamclass for seamless handling of zip-files!
But beware that there's a small bug in file ZIP.cpp, consider: Lines 191ff. read
char* buf=new char[std::max(comment_length,std::max(filename_length,extra_length))];
istream.read(buf,filename_length);
buf[filename_length]=0;
This neglects the fact that the buffer needs one more char space for the terminator (line 193), possibly corrupting the heap! Thus line 191 should read
char* buf=new char[std::max(comment_length,std::max(static_cast<unsigned short>(filename_length+1),extra_length))];
So, if you're going to use this class you should consider fixing this bug. It's no hassle.
I'm basically looking for a C++ version of fdopen(). I did a bit of research on this and it is one of those things that seems like it should be easy, but turns out to be very complicated. Am I missing something in this belief (i.e. it really is easy)? If not, is there a good library out there somewhere to handle this?
EDIT: Moved my example solution to a separate answer.
From the answer given by Éric Malenfant:
AFAIK, there is no way to do this in
standard C++. Depending on your
platform, your implementation of the
standard library may offer (as a
nonstandard extension) a fstream
constructor taking a file descriptor
as input. (This is the case for
libstdc++, IIRC) or a FILE*.
Based on above observations and my research below there's working code in two variants; one for libstdc++ and another one for Microsoft Visual C++.
libstdc++
There's non-standard __gnu_cxx::stdio_filebuf class template which inherits std::basic_streambuf and has the following constructor
stdio_filebuf (int __fd, std::ios_base::openmode __mode, size_t __size=static_cast< size_t >(BUFSIZ))
with description This constructor associates a file stream buffer with an open POSIX file descriptor.
We create it passing POSIX handle (line 1) and then we pass it to istream's constructor as basic_streambuf (line 2):
#include <ext/stdio_filebuf.h>
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main()
{
ofstream ofs("test.txt");
ofs << "Writing to a basic_ofstream object..." << endl;
ofs.close();
int posix_handle = fileno(::fopen("test.txt", "r"));
__gnu_cxx::stdio_filebuf<char> filebuf(posix_handle, std::ios::in); // 1
istream is(&filebuf); // 2
string line;
getline(is, line);
cout << "line: " << line << std::endl;
return 0;
}
Microsoft Visual C++
There used to be non-standard version of ifstream's constructor taking POSIX file descriptor but it's missing both from current docs and from code. There is another non-standard version of ifstream's constructor taking FILE*
explicit basic_ifstream(_Filet *_File)
: _Mybase(&_Filebuffer),
_Filebuffer(_File)
{ // construct with specified C stream
}
and it's not documented (I couldn't even find any old documentation where it would be present). We call it (line 1) with the parameter being the result of calling _fdopen to get C stream FILE* from POSIX file handle.
#include <cstdio>
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main()
{
ofstream ofs("test.txt");
ofs << "Writing to a basic_ofstream object..." << endl;
ofs.close();
int posix_handle = ::_fileno(::fopen("test.txt", "r"));
ifstream ifs(::_fdopen(posix_handle, "r")); // 1
string line;
getline(ifs, line);
ifs.close();
cout << "line: " << line << endl;
return 0;
}
AFAIK, there is no way to do this in standard C++. Depending on your platform, your implementation of the standard library may offer (as a nonstandard extension) a fstream constructor taking a file descriptor (This is the case for libstdc++, IIRC) or a FILE* as an input.
Another alternative would be to use a boost::iostreams::file_descriptor device, which you could wrap in a boost::iostreams::stream if you want to have an std::stream interface to it.
There's a good chance your compiler offers a FILE-based fstream constructor, even though it's non-standard. For example:
FILE* f = fdopen(my_fd, "a");
std::fstream fstr(f);
fstr << "Greetings\n";
But as far as I know, there's no portable way to do this.
Part of the original (unstated) motivation of this question is to have the ability to pass data either between programs or between two parts of a test program using a safely created temporary file, but tmpnam() throws a warning in gcc, so I wanted to use mkstemp() instead. Here is a test program that I wrote based on the answer given by Éric Malenfant but using mkstemp() instead of fdopen(); this works on my Ubuntu system with Boost libraries installed:
#include <stdlib.h>
#include <string.h>
#include <assert.h>
#include <string>
#include <iostream>
#include <boost/filesystem.hpp>
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/iostreams/stream.hpp>
using boost::iostreams::stream;
using boost::iostreams::file_descriptor_sink;
using boost::filesystem::path;
using boost::filesystem::exists;
using boost::filesystem::status;
using boost::filesystem::remove;
int main(int argc, const char *argv[]) {
char tmpTemplate[13];
strncpy(tmpTemplate, "/tmp/XXXXXX", 13);
stream<file_descriptor_sink> tmp(mkstemp(tmpTemplate));
assert(tmp.is_open());
tmp << "Hello mkstemp!" << std::endl;
tmp.close();
path tmpPath(tmpTemplate);
if (exists(status(tmpPath))) {
std::cout << "Output is in " << tmpPath.file_string() << std::endl;
std::string cmd("cat ");
cmd += tmpPath.file_string();
system(cmd.c_str());
std::cout << "Removing " << tmpPath.file_string() << std::endl;
remove(tmpPath);
}
}
It actually is quite easy. Nicolai M. Josuttis has released fdstream in conjunction with his book The C++ Standard Library - A Tutorial and Reference. You can find the 184 line implementation here.
I've tried the solution proposed above for libstdc++ by Piotr Dobrogost, and found that it had a painful flaw: Due to the lack of a proper move constructor for istream, it's very difficult to get the newly constructed istream object out of the creating function. Another issue with it is that it leaks a FILE object (even thought not the underlying posix file descriptor). Here's an alternative solution that avoids these issues:
#include <fstream>
#include <string>
#include <ext/stdio_filebuf.h>
#include <type_traits>
bool OpenFileForSequentialInput(ifstream& ifs, const string& fname)
{
ifs.open(fname.c_str(), ios::in);
if (! ifs.is_open()) {
return false;
}
using FilebufType = __gnu_cxx::stdio_filebuf<std::ifstream::char_type>;
static_assert( std::is_base_of<ifstream::__filebuf_type, FilebufType>::value &&
(sizeof(FilebufType) == sizeof(ifstream::__filebuf_type)),
"The filebuf type appears to have extra data members, the cast might be unsafe");
const int fd = static_cast<FilebufType*>(ifs.rdbuf())->fd();
assert(fd >= 0);
if (0 != posix_fadvise(fd, 0, 0, POSIX_FADV_SEQUENTIAL)) {
ifs.close();
return false;
}
return true;
}
The call to posix_fadvise() demonstrates a potential use. Also note that the example uses static_assert and using which are C++ 11, other than that it should build just fine in C++ 03 mode.
Another non-portable solution is to use mmap (or its Windows' analogue) and then construct std::iostream from a pointer that mmap gave like so.
Yeah, it does not construct exactly an std::fstream, but this requirement rarely needs to be met because every piece of code should depend on stream interfaces (e.g. std::istream) rather than on their implementations.
I think this solution is more portable than use of STL implementation-specific hacks, because this way you only depend on an operating system, rather than on a specific implementation of STL for the same OS.
My understanding is that there is no association with FILE pointers or file descriptors in the C++ iostream object model in order to keep code portable.
That said, I saw several places refer to the mds-utils or boost to help bridge that gap.