I've written my own specialisation of each virtual member function of std::ctype<char16_t>, so that this now works:
#include <string>
#include <locale>
#include "char16_facets.h" // Header containing my ctype specialisation
#include <sstream>
#include <iostream>
// Implemented elsewhere using iconv
std::string Convert(std::basic_string<char16_t>);
int main() {
std::basic_string<char16_t> s("Hello, world.");
std::basic_stringstream<char16_t> ss(s);
ss.imbue(std::locale(ss.getloc(), new std::ctype<char16_t>()));
std::basic_string<char16_t> t;
ss >> t;
std::cout << Convert(t) << " ";
ss >> t;
std::cout << Convert(t) << std::endl;
}
Is there a way to make streams use the new ctype specialisation by default, so I don't have to imbue each stream with a new locale?
I haven't written a new class, just provided
template<>
inline bool std::ctype<char16_t>::do_is (std::ctype_base::mask m, char16_t c) const {
etc. I'd sort of hoped it would be picked up automatically, so long as it was declared before I #include <sstream> but it isn't.
Most of the work for the above was done using G++ and libstdc++ 4.8, but I get the same result with them built from SVN trunk.
Edit - Update This question originally asked about how to get number extraction working. However, given a stream imbued with correct ctype and numpunct implementations, then no specialisation of num_get is necessary; simply
ss.imbue(std::locale(ss.getloc(), new std::num_get<char16_t>()));
and it will work, with either gcc version.
Again, is there some way to get the streams to pick this up automatically, rather than having to imbue every stream with it?
Use std::locale::global():
std::locale::global(std::locale(std::locale(), new std::ctype<char16_t>()));
Related
I'm basically looking for a C++ version of fdopen(). I did a bit of research on this and it is one of those things that seems like it should be easy, but turns out to be very complicated. Am I missing something in this belief (i.e. it really is easy)? If not, is there a good library out there somewhere to handle this?
EDIT: Moved my example solution to a separate answer.
From the answer given by Éric Malenfant:
AFAIK, there is no way to do this in
standard C++. Depending on your
platform, your implementation of the
standard library may offer (as a
nonstandard extension) a fstream
constructor taking a file descriptor
as input. (This is the case for
libstdc++, IIRC) or a FILE*.
Based on above observations and my research below there's working code in two variants; one for libstdc++ and another one for Microsoft Visual C++.
libstdc++
There's non-standard __gnu_cxx::stdio_filebuf class template which inherits std::basic_streambuf and has the following constructor
stdio_filebuf (int __fd, std::ios_base::openmode __mode, size_t __size=static_cast< size_t >(BUFSIZ))
with description This constructor associates a file stream buffer with an open POSIX file descriptor.
We create it passing POSIX handle (line 1) and then we pass it to istream's constructor as basic_streambuf (line 2):
#include <ext/stdio_filebuf.h>
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main()
{
ofstream ofs("test.txt");
ofs << "Writing to a basic_ofstream object..." << endl;
ofs.close();
int posix_handle = fileno(::fopen("test.txt", "r"));
__gnu_cxx::stdio_filebuf<char> filebuf(posix_handle, std::ios::in); // 1
istream is(&filebuf); // 2
string line;
getline(is, line);
cout << "line: " << line << std::endl;
return 0;
}
Microsoft Visual C++
There used to be non-standard version of ifstream's constructor taking POSIX file descriptor but it's missing both from current docs and from code. There is another non-standard version of ifstream's constructor taking FILE*
explicit basic_ifstream(_Filet *_File)
: _Mybase(&_Filebuffer),
_Filebuffer(_File)
{ // construct with specified C stream
}
and it's not documented (I couldn't even find any old documentation where it would be present). We call it (line 1) with the parameter being the result of calling _fdopen to get C stream FILE* from POSIX file handle.
#include <cstdio>
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main()
{
ofstream ofs("test.txt");
ofs << "Writing to a basic_ofstream object..." << endl;
ofs.close();
int posix_handle = ::_fileno(::fopen("test.txt", "r"));
ifstream ifs(::_fdopen(posix_handle, "r")); // 1
string line;
getline(ifs, line);
ifs.close();
cout << "line: " << line << endl;
return 0;
}
AFAIK, there is no way to do this in standard C++. Depending on your platform, your implementation of the standard library may offer (as a nonstandard extension) a fstream constructor taking a file descriptor (This is the case for libstdc++, IIRC) or a FILE* as an input.
Another alternative would be to use a boost::iostreams::file_descriptor device, which you could wrap in a boost::iostreams::stream if you want to have an std::stream interface to it.
There's a good chance your compiler offers a FILE-based fstream constructor, even though it's non-standard. For example:
FILE* f = fdopen(my_fd, "a");
std::fstream fstr(f);
fstr << "Greetings\n";
But as far as I know, there's no portable way to do this.
Part of the original (unstated) motivation of this question is to have the ability to pass data either between programs or between two parts of a test program using a safely created temporary file, but tmpnam() throws a warning in gcc, so I wanted to use mkstemp() instead. Here is a test program that I wrote based on the answer given by Éric Malenfant but using mkstemp() instead of fdopen(); this works on my Ubuntu system with Boost libraries installed:
#include <stdlib.h>
#include <string.h>
#include <assert.h>
#include <string>
#include <iostream>
#include <boost/filesystem.hpp>
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/iostreams/stream.hpp>
using boost::iostreams::stream;
using boost::iostreams::file_descriptor_sink;
using boost::filesystem::path;
using boost::filesystem::exists;
using boost::filesystem::status;
using boost::filesystem::remove;
int main(int argc, const char *argv[]) {
char tmpTemplate[13];
strncpy(tmpTemplate, "/tmp/XXXXXX", 13);
stream<file_descriptor_sink> tmp(mkstemp(tmpTemplate));
assert(tmp.is_open());
tmp << "Hello mkstemp!" << std::endl;
tmp.close();
path tmpPath(tmpTemplate);
if (exists(status(tmpPath))) {
std::cout << "Output is in " << tmpPath.file_string() << std::endl;
std::string cmd("cat ");
cmd += tmpPath.file_string();
system(cmd.c_str());
std::cout << "Removing " << tmpPath.file_string() << std::endl;
remove(tmpPath);
}
}
It actually is quite easy. Nicolai M. Josuttis has released fdstream in conjunction with his book The C++ Standard Library - A Tutorial and Reference. You can find the 184 line implementation here.
I've tried the solution proposed above for libstdc++ by Piotr Dobrogost, and found that it had a painful flaw: Due to the lack of a proper move constructor for istream, it's very difficult to get the newly constructed istream object out of the creating function. Another issue with it is that it leaks a FILE object (even thought not the underlying posix file descriptor). Here's an alternative solution that avoids these issues:
#include <fstream>
#include <string>
#include <ext/stdio_filebuf.h>
#include <type_traits>
bool OpenFileForSequentialInput(ifstream& ifs, const string& fname)
{
ifs.open(fname.c_str(), ios::in);
if (! ifs.is_open()) {
return false;
}
using FilebufType = __gnu_cxx::stdio_filebuf<std::ifstream::char_type>;
static_assert( std::is_base_of<ifstream::__filebuf_type, FilebufType>::value &&
(sizeof(FilebufType) == sizeof(ifstream::__filebuf_type)),
"The filebuf type appears to have extra data members, the cast might be unsafe");
const int fd = static_cast<FilebufType*>(ifs.rdbuf())->fd();
assert(fd >= 0);
if (0 != posix_fadvise(fd, 0, 0, POSIX_FADV_SEQUENTIAL)) {
ifs.close();
return false;
}
return true;
}
The call to posix_fadvise() demonstrates a potential use. Also note that the example uses static_assert and using which are C++ 11, other than that it should build just fine in C++ 03 mode.
Another non-portable solution is to use mmap (or its Windows' analogue) and then construct std::iostream from a pointer that mmap gave like so.
Yeah, it does not construct exactly an std::fstream, but this requirement rarely needs to be met because every piece of code should depend on stream interfaces (e.g. std::istream) rather than on their implementations.
I think this solution is more portable than use of STL implementation-specific hacks, because this way you only depend on an operating system, rather than on a specific implementation of STL for the same OS.
My understanding is that there is no association with FILE pointers or file descriptors in the C++ iostream object model in order to keep code portable.
That said, I saw several places refer to the mds-utils or boost to help bridge that gap.
I need to use Boost.Serialization to serialize floating-point numbers. Since NaN and infinites cannot natively be read from an input stream, I am trying to use the facets in boost/math/special_functions. I have tested them on my platform using code similar to the examples we can find here: http://www.boost.org/doc/libs/1_50_0/libs/math/doc/sf_and_dist/html/math_toolkit/utils/fp_facets/intro.html
However, the following code still fails to properly unserialize non-finite floating point values (an exception is thrown with description "input stream error").
#include <limits>
#include <locale>
#include <sstream>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/archive/xml_oarchive.hpp>
#include <boost/math/special_functions/nonfinite_num_facets.hpp>
#include <boost/serialization/nvp.hpp>
struct Data {
float f;
Data() : f(std::numeric_limits<float>::quiet_NaN()) {}
template <class Archive>
void serialize(Archive & ar, unsigned const)
{
ar & BOOST_SERIALIZATION_NVP(f);
}
};
void test()
{
using namespace boost::archive;
Data d;
std::ostringstream oss;
xml_oarchive oar(oss);
oar << BOOST_SERIALIZATION_NVP(d);
//std::cout << oss.str() << std::endl;
std::istringstream iss(oss.str());
std::locale const new_loc(iss.getloc(), new boost::math::nonfinite_num_get<char>);
iss.imbue(new_loc);
xml_iarchive iar(iss);
iar >> BOOST_SERIALIZATION_NVP(d);
std::cout << d.f << std::endl;
}
Am I doing something wrong? Is there a problem with my Boost version or my platform? Is there a better solution? Any help would be greatly appreciated.
I have found the solution by reading the following implementation note:
http://www.boost.org/doc/libs/1_55_0/libs/serialization/doc/implementation.html#charencoding
When constructing an archive with the default flag, the stream's locale is changed to address character encoding issues, but this mechanism can be disabled using flag boost::archive::no_codecvt.
If I replace the line
xml_iarchive iar(iss);
with
xml_iarchive iar(iss, no_codecvt);
then it works.
When using ifstream class to read words from an input file, I have used the following expression:
#include <iostream>
#include <fstream>
int main(int argc, char *argv[])
{
std::ifstream inputStream(myFile.txt);
std::string myString;
myFile.open()
while(myFile.good())
{
myFile >> myString;
printf("%s \n", myString);
}
return 0;
}
The contents of myFile.txt are:
" This is a simple program. "
The compiles and executes as expected using g++ compiler.
However, the same code when compiled using msvc 2008, returns error at the extraction operator (>>) requiring me to replace the std::string with either an initialized character array or any of the supported native types.
This threw me off as I was expecting the usage of the standard library to be same across implementations.
I understand the compile error and know the way to fix it via using c_str().
But, it would help me a great deal, if someone could clarify why the usage for the standard library is different across platforms.
To me it is not starndard anymore !!
EDIT: Code updated to be complete. Content of myFile.txt updated.
Chances are that you forgot to #include <string>. Without it, Microsoft's version of <iostream> (and such) include enough of a declaration of std::string for some things to work, but other parts are missing, so you get strange, seemingly inexplicable failures.
One of the things that's missing is most of the operator overloads for std::string, which is exactly what you seem to be missing.
As an aside, while (myfile.good()) ... is pretty much a guaranteed bug -- you probably want:
while (myfile>>myString)
std::cout << myString << " \n";
Alternatively, you could do the job with a standard algorithm:
#include <string>
#include <algorithm>
#include <iterator>
#include <fstream>
#include <iostream>
int main() {
std::ifstream myfile("input.txt");
std::copy(std::istream_iterator<std::string>(myfile),
std::istream_iterator<std::string>(),
std::ostream_iterator<std::string>(std::cout, " \n"));
return 0;
}
The following compiles fine for me on MSVC 2010:
std::ifstream inputStream;
std::string myString;
inputStream.open("myFile.txt", std::ifstream::in);
while(inputStream.good())
{
inputStream >> myString;
}
Note: without using std::ifstream::in as my open mode, I got the same error as you. I suggest you check what value you have for this parameter.
I'm basically looking for a C++ version of fdopen(). I did a bit of research on this and it is one of those things that seems like it should be easy, but turns out to be very complicated. Am I missing something in this belief (i.e. it really is easy)? If not, is there a good library out there somewhere to handle this?
EDIT: Moved my example solution to a separate answer.
From the answer given by Éric Malenfant:
AFAIK, there is no way to do this in
standard C++. Depending on your
platform, your implementation of the
standard library may offer (as a
nonstandard extension) a fstream
constructor taking a file descriptor
as input. (This is the case for
libstdc++, IIRC) or a FILE*.
Based on above observations and my research below there's working code in two variants; one for libstdc++ and another one for Microsoft Visual C++.
libstdc++
There's non-standard __gnu_cxx::stdio_filebuf class template which inherits std::basic_streambuf and has the following constructor
stdio_filebuf (int __fd, std::ios_base::openmode __mode, size_t __size=static_cast< size_t >(BUFSIZ))
with description This constructor associates a file stream buffer with an open POSIX file descriptor.
We create it passing POSIX handle (line 1) and then we pass it to istream's constructor as basic_streambuf (line 2):
#include <ext/stdio_filebuf.h>
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main()
{
ofstream ofs("test.txt");
ofs << "Writing to a basic_ofstream object..." << endl;
ofs.close();
int posix_handle = fileno(::fopen("test.txt", "r"));
__gnu_cxx::stdio_filebuf<char> filebuf(posix_handle, std::ios::in); // 1
istream is(&filebuf); // 2
string line;
getline(is, line);
cout << "line: " << line << std::endl;
return 0;
}
Microsoft Visual C++
There used to be non-standard version of ifstream's constructor taking POSIX file descriptor but it's missing both from current docs and from code. There is another non-standard version of ifstream's constructor taking FILE*
explicit basic_ifstream(_Filet *_File)
: _Mybase(&_Filebuffer),
_Filebuffer(_File)
{ // construct with specified C stream
}
and it's not documented (I couldn't even find any old documentation where it would be present). We call it (line 1) with the parameter being the result of calling _fdopen to get C stream FILE* from POSIX file handle.
#include <cstdio>
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main()
{
ofstream ofs("test.txt");
ofs << "Writing to a basic_ofstream object..." << endl;
ofs.close();
int posix_handle = ::_fileno(::fopen("test.txt", "r"));
ifstream ifs(::_fdopen(posix_handle, "r")); // 1
string line;
getline(ifs, line);
ifs.close();
cout << "line: " << line << endl;
return 0;
}
AFAIK, there is no way to do this in standard C++. Depending on your platform, your implementation of the standard library may offer (as a nonstandard extension) a fstream constructor taking a file descriptor (This is the case for libstdc++, IIRC) or a FILE* as an input.
Another alternative would be to use a boost::iostreams::file_descriptor device, which you could wrap in a boost::iostreams::stream if you want to have an std::stream interface to it.
There's a good chance your compiler offers a FILE-based fstream constructor, even though it's non-standard. For example:
FILE* f = fdopen(my_fd, "a");
std::fstream fstr(f);
fstr << "Greetings\n";
But as far as I know, there's no portable way to do this.
Part of the original (unstated) motivation of this question is to have the ability to pass data either between programs or between two parts of a test program using a safely created temporary file, but tmpnam() throws a warning in gcc, so I wanted to use mkstemp() instead. Here is a test program that I wrote based on the answer given by Éric Malenfant but using mkstemp() instead of fdopen(); this works on my Ubuntu system with Boost libraries installed:
#include <stdlib.h>
#include <string.h>
#include <assert.h>
#include <string>
#include <iostream>
#include <boost/filesystem.hpp>
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/iostreams/stream.hpp>
using boost::iostreams::stream;
using boost::iostreams::file_descriptor_sink;
using boost::filesystem::path;
using boost::filesystem::exists;
using boost::filesystem::status;
using boost::filesystem::remove;
int main(int argc, const char *argv[]) {
char tmpTemplate[13];
strncpy(tmpTemplate, "/tmp/XXXXXX", 13);
stream<file_descriptor_sink> tmp(mkstemp(tmpTemplate));
assert(tmp.is_open());
tmp << "Hello mkstemp!" << std::endl;
tmp.close();
path tmpPath(tmpTemplate);
if (exists(status(tmpPath))) {
std::cout << "Output is in " << tmpPath.file_string() << std::endl;
std::string cmd("cat ");
cmd += tmpPath.file_string();
system(cmd.c_str());
std::cout << "Removing " << tmpPath.file_string() << std::endl;
remove(tmpPath);
}
}
It actually is quite easy. Nicolai M. Josuttis has released fdstream in conjunction with his book The C++ Standard Library - A Tutorial and Reference. You can find the 184 line implementation here.
I've tried the solution proposed above for libstdc++ by Piotr Dobrogost, and found that it had a painful flaw: Due to the lack of a proper move constructor for istream, it's very difficult to get the newly constructed istream object out of the creating function. Another issue with it is that it leaks a FILE object (even thought not the underlying posix file descriptor). Here's an alternative solution that avoids these issues:
#include <fstream>
#include <string>
#include <ext/stdio_filebuf.h>
#include <type_traits>
bool OpenFileForSequentialInput(ifstream& ifs, const string& fname)
{
ifs.open(fname.c_str(), ios::in);
if (! ifs.is_open()) {
return false;
}
using FilebufType = __gnu_cxx::stdio_filebuf<std::ifstream::char_type>;
static_assert( std::is_base_of<ifstream::__filebuf_type, FilebufType>::value &&
(sizeof(FilebufType) == sizeof(ifstream::__filebuf_type)),
"The filebuf type appears to have extra data members, the cast might be unsafe");
const int fd = static_cast<FilebufType*>(ifs.rdbuf())->fd();
assert(fd >= 0);
if (0 != posix_fadvise(fd, 0, 0, POSIX_FADV_SEQUENTIAL)) {
ifs.close();
return false;
}
return true;
}
The call to posix_fadvise() demonstrates a potential use. Also note that the example uses static_assert and using which are C++ 11, other than that it should build just fine in C++ 03 mode.
Another non-portable solution is to use mmap (or its Windows' analogue) and then construct std::iostream from a pointer that mmap gave like so.
Yeah, it does not construct exactly an std::fstream, but this requirement rarely needs to be met because every piece of code should depend on stream interfaces (e.g. std::istream) rather than on their implementations.
I think this solution is more portable than use of STL implementation-specific hacks, because this way you only depend on an operating system, rather than on a specific implementation of STL for the same OS.
My understanding is that there is no association with FILE pointers or file descriptors in the C++ iostream object model in order to keep code portable.
That said, I saw several places refer to the mds-utils or boost to help bridge that gap.
May I know how I can convert std::string, to MSVC specific __int64?
_atoi64, _atoi64_l, _wtoi64, _wtoi64_l
std::string str = "1234";
__int64 v =_atoi64(str.c_str());
See also this link (although it is for linux/unix): Why doesn't C++ reimplement C standard functions with C++ elements/style?
Here's one way:
#include <iostream>
#include <string>
#include <sstream>
using namespace std;
int main() {
string s( "1234567890987654321");
stringstream strm( s);
__int64 x;
strm >> x;
cout << x;
}
__int64, while an extension, is still just a numeric type. Use whichever method you would typically use.
Boost lexical cast is my favorite. It pretty much wraps up Michaels answer in an easy to use form:
__int64 x = boost::lexical_cast<__int64>("3473472936");
If you can't use boost, you can still do a pretty good job of making a simple version. Here's an implementation I wrote for another answer:
template <typename R>
const R lexical_cast(const std::string& s)
{
std::stringstream ss(s);
R result;
if ((ss >> result).fail() || !(ss >> std::ws).eof())
{
throw std::bad_cast();
}
return result;
}
It does some extras, like checking for trailing characters. ("123125asd" would fail). If the cast cannot be made, bad_cast is thrown. (Similar to boost.)
Also, if you have access to boost, you can avoid the need to use the MSVC-specific __int64 extension with:
#include <boost/cstdint.hpp>
typedef boost::int64_t int64;
To get int64 on any platform that provides it, without changing your code.