I have the following code and it works pretty good (other than the fact that it's pretty slow, but I don't care much about that). It doesn't seem intuitive that this would write the entire contents of the infile to the outfile.
// Returns 1 if failed and 0 if successful
int WriteFileContentsToNewFile(string inFilename, string outFilename)
{
ifstream infile(inFilename.c_str(), ios::binary);
ofstream outfile(outFilename.c_str(), ios::binary);
if( infile.is_open() && outfile.is_open() && infile.good() && outfile.good() )
{
outfile << infile.rdbuf();
outfile.close();
infile.close();
}
else
return 1;
return 0;
}
Any insight?
iostream classes are just wrappers around I/O buffers. The iostream itself doesn't do a whole lot… mainly, it the provides operator>> formatting operators. The buffer is provided by an object derived from basic_streambuf, which you can get and set using rdbuf().
basic_streambuf is an abstract base with a number of virtual functions which are overridden to provide a uniform interface for reading/writing files, strings, etc. The function basic_ostream<…>::operator<<( basic_streambuf<…> ) is defined to keep reading through the buffer until the underlying data source is exhausted.
iostream is a terrible mess, though.
Yes, it's specified in the standard and it's actually quite simple. rdbuf() just returns a pointer to the underlying basic_streambuf object for the given [io]stream object.
basic_ostream<...> has an overload for operator<< for a pointer to basic_streambuf<...> which writes out the contents of the basic_streambuf<...>.
A quick look at the source code shows that basic_ofstream is the wrapper around basic_filebuf.
Related
I'm learning file handling in c++ from internet alone. I came across the read and write function. But the parameters they take confused me.
So, I found the syntax as
fstream fout;
fout.write( (char *) &obj, sizeof(obj) );
and
fstream fin;
fin.read( (char *) &obj, sizeof(obj) );
In both of these, what is the function of char*?
And how does it read and write the file?
The function fstream::read has the following function signature:
istream& read (char* s, streamsize n);
You need to cast your arguments to the correct type. (char*) tells the compiler to pretend &obj is the correct type. Usually, this is a really bad idea.
Instead, you should do it this way:
// C++ program to demonstrate getline() function
#include <iostream>
#include <string>
using namespace std;
int main()
{
string str;
fstream fin;
getline(fin, str); // use cin instead to read from stdin
return 0;
}
Source: https://www.geeksforgeeks.org/getline-string-c/
The usage of the char * cast with read and write is to treat the obj variable as generic, continuous, characters (ignoring any structure).
The read function will read from the stream directly into the obj variable, without any byte translation or mapping to data members (fields). Note, pointers in classes or structures will be replaced with whatever value comes from the stream (which means the pointer will probably point to an invalid or improper location). Beware of padding issues.
The write function will the entire area of memory, occupied by obj, to the stream. Any padding between structure or class members will also be written. Values of pointers will be written to the stream, not the item that the pointer points to.
Note: these functions work "as-is". There are no conversions or translations of the data. For example, no conversion between Big Endain and Little Endian; no processing of the "end of line" or "end of file" characters. Basically mirror image data transfers.
I am attempting to use libpng in order to read a png from
a Qt resource. The catch: The class doing the reading
should not have any dependencies of Qt.
In a first step, reading http://www.piko3d.net/tutorials/libpng-tutorial-loading-png-files-from-streams/#CustomRead I already succeeded in writing a function
read_png(istream& in)
I also succeeded in passing a plain old ifstream
ifstream in("abs_path_to_png/icon.png");
to read_png(..) and having it successfully reading the png. But how to get
a (preferably platform independent) istream from a Qt resource? Performance
is no great issue so I initially came up with
bool Io_Qt::get_istringstream_from_QFile(QFile& qfile, istringstream& iss)
{
// [.. Some checking for existence and the file being open ..]
QString qs(qfile.readAll());
iss.str(qs.toStdString());
// I also tried: QByteArray arr(qfile.readAll()); iss.str(arr.data());
return qfile.isOpen();
}
// Someplace else iss and qfile are created like this:
istringstream iss(std::stringstream::in | std::stringstream::binary);
QFile qfile(":/res/icon.png");
qfile.open(QIODevice::ReadOnly);
This in fact yields an iss that is, at first glance, looking good, when saying
cout << "'" << iss.str().c_str() << "'" << endl;
I get
'�PNG
'
There appears to be some whitespace issue though. For
ifstream in("abs_path_to_png/icon.png");
char c;
cout << "'";
for (int j=0;j<8;j++)
{
in >> c;
cout << c;
}
cout << "'" << endl;
yields
'�PNG'
and while the latter works the former variation ultimately leads the libpng checking function png_sig_cmp(..) into rejecting my png as invalid. My first reflex is about "binary". However:
istringstream iss(std::stringstream::in | std::stringstream::binary); feels right.
QIODevice::ReadOnly does not appear to have a binary partner.
Do you see what I missed?
You're working with the streams like they're text data with lexical extraction operators. Check out ios::binary as well as the read and write methods which are appropriate when working with a binary stream.
I would forgo operator<< and operator>> outright in your case in favor of read and write. Use ostream::write to write the byte array data returned from QIODevice::readAll() to transfer its contents to your temporary stringstream, e.g., and use ostream::read in your tests to validate its contents.
A good test case to make sure you transferred properly is to write a test where you read the contents from a QFile, use ostream::write to transfer it to an binary output file stream (ofstream), and then try to load it up in an image software to see if it's okay. Then swap your file stream with a stringstream and pass it to libpng when you have that working.
As Ike says, it seems indeed to be about the differences between
text-centered operators '>>', '<<' and stuff like '.str(..)' as opposed
to binary-centered commands like '.read', and '.write'. Plus it is
about initializing the streams correctly. When I finally got the program
to do what I wanted the gospel went something like this:
First I used a plain stringstream alongside the QFile:
// Explicitly setting flags should at least contain ::in and ::out
// stringstream ss(std::stringstream::in | std::stringstream::out | std::stringstream::binary)
// However, the default constructor is perfectly fine.
stringstream ss;
QFile qfile(":/res/icon.png");
qfile.open(QIODevice::ReadOnly);
This I passed to my function which now looks like this:
bool Io_Qt::get_stringstream_from_QFile(QFile& qfile, stringstream& ss)
{
// [.. some sanity checks..]
QDataStream in(&qfile);
uint len = qfile.size();
char* c = (char*)malloc(len*sizeof(char));
in.readRawData(c,len);
ss.write(c,len);
free (c);
return true;
}
This stream was filled, and had the right size. Especially since
.write(..) writes the required number of characters regardless
of how many zeros are within the data. My biggest problem was
my being loath to have both std::stringstream::in AND
std::stringstream::out activated at the same time because the
combination seemed somewhat wacky to me. Yet both are needed.
However, I found I may skip std::stringstream::binary.
But since it does not seem to do any harm I like to
keep it for good luck. Feel free to comment on this superstition though! :-)
A more clean, less C-ish, more Qt/C++ -ish version can be:
QFile file(filePath);
file.open(QIODevice::ReadOnly);
QByteArray data = file.readAll();
std::istringstream iss(data.toStdString());
now use iss, in my case this was for libTIFF:
TIFF* tif = TIFFStreamOpen("MemTIFF", &iss);
// ...
Also, for PNGs you can now follow your already posted article, since std::istringstream is of type std::istream.
Note, this solution involves full loading of the file data into memory.
So I am writing my own custom FTP client for a school project. I managed to get everything to work with the swarming FTP client and am down to one last small part...reading the .part files into the main file. I need to do two things. (1) Get this to read each file and write to the final file properly (2) The command to delete the part files after I am done with each one.
Can someone please help me to fix my concatenate function I wrote below? I thought I had it right to read each file until the EOF and then go on to the next.
In this case *numOfThreads is 17. Ended up with a file of 4742442 bytes instead of 594542592 bytes. Thanks and I am happy to provide any other useful information.
EDIT: Modified code for comment below.
std::string s = "Fedora-15-x86_64-Live-Desktop.iso";
std::ofstream out;
out.open(s.c_str(), std::ios::out);
for (int i = 0; i < 17; ++i)
{
std::ifstream in;
std::ostringstream convert;
convert << i;
std::string t = s + ".part" + convert.str();
in.open(t.c_str(), std::ios::in | std::ios::binary);
int size = 32*1024;
char *tempBuffer = new char[size];
if (in.good())
{
while (in.read(tempBuffer, size))
out.write(tempBuffer, in.gcount());
}
delete [] tempBuffer;
in.close();
}
out.close();
return 0;
Almost everything in your copying loop has problems.
while (!in.eof())
This is broken. Not much more to say than that.
bzero(tempBuffer, size);
This is fairly harmless, but utterly pointless.
in.read(tempBuffer, size);
This the "almost" part -- i.e., the one piece that isn't obviously broken.
out.write(tempBuffer, strlen(tempBuffer));
You don't want to use strlen to determine the length -- it's intended only for NUL-terminated (C-style) strings. If (as is apparently the case) the data you read may contain zero-bytes (rather than using zero-bytes only to signal the end of a string), this will simply produce the wrong size.
What you normally want to do is a loop something like:
while (read(some_amount) == succeeded)
write(amount that was read);
In C++ that will typically be something like:
while (infile.read(buffer, buffer_size))
outfile.write(buffer, infile.gcount());
It's probably also worth noting that since you're allocating memory for the buffer using new, but never using delete, your function is leaking memory. Probably better to do without new for this -- an array or vector would be obvious alternatives here.
Edit: as for why while (infile.read(...)) works, the read returns a reference to the stream. The stream in turn provides a conversion to bool (in C++11) or void * (in C++03) that can be interpreted as a Boolean. That conversion operator returns the state of the stream, so if reading failed, it will be interpreted as false, but as long as it succeeded, it will be interpreted as true.
I am trying to achieve something like this:
while (ifstream has not been entirely read)
{
read a chunk of data into a buffer that has size BUFLEN
write this buffer to ostream
}
At first I tried to achieve this by using ifstream.eof() as my while condition, but I've heard that this is not the way to go. I've been looking at std::ios::ifstream's other functions, but can't figure out what else to use.
PS: I am using a buffer because the file that is being transferred can get very big.
The iostream classes take care of all necessary buffering, so you don't
have to. The usual idiom to copy an entire file is just:
fout << fin.rdbuf();
iostream takes care of all of the necessary buffering. (This is a
somewhat unusual use of <<, since it doesn't format. Historical
reasons, no doubt.)
If you need the loop, perhaps because you want to do some
transformations on the data before rewriting it, then it's a little
tricker, since istream::read “fails” unless it reads the
requested number of characters. Because of this, you have to also check
how many characters were read, and process them even if the read failed:
int readCount;
while ( fin.read( &buf[0], buf.size() )
|| (readCount = fin.gcount()) != 0 ) {
// ...
fout.write( &buf[0], readCount );
}
This is fairly ugly; a better solution might be to wrap the buffer in a
class, and define an operator<< for this class.
The istream::read function returns the stream, which can be used as a boolean expression, so you can do something like:
while (is.read(buffer, BUFLEN))
{
outputfile.write(buffer, is.gcount());
}
if (is.eof())
{
if (is.gcount() > 0)
{
// Still a few bytes left to write
outputfile.write(buffer, is.gcount());
}
}
else if (is.bad())
{
// Error reading
}
You might want to check that the write inside the loop doesn't fail too.
Your logic is simply wrong.
You need to go about it more like this:
while (a chunk larger than zero could be read)
{
write chunk to output
}
See how that is even simpler? There's no need to explicitly check for "end of file", just read data until you fail. Then you're done.
I have an object with several text strings as members. I want to write this object to the file all at once, instead of writing each string to file. How can I do that?
You can override operator>> and operator<< to read/write to stream.
Example Entry struct with some values:
struct Entry2
{
string original;
string currency;
Entry2() {}
Entry2(string& in);
Entry2(string& original, string& currency)
: original(original), currency(currency)
{}
};
istream& operator>>(istream& is, Entry2& en);
ostream& operator<<(ostream& os, const Entry2& en);
Implementation:
using namespace std;
istream& operator>>(istream& is, Entry2& en)
{
is >> en.original;
is >> en.currency;
return is;
}
ostream& operator<<(ostream& os, const Entry2& en)
{
os << en.original << " " << en.currency;
return os;
}
Then you open filestream, and for each object you call:
ifstream in(filename.c_str());
Entry2 e;
in >> e;
//if you want to use read:
//in.read(reinterpret_cast<const char*>(&e),sizeof(e));
in.close();
Or output:
Entry2 e;
// set values in e
ofstream out(filename.c_str());
out << e;
out.close();
Or if you want to use stream read and write then you just replace relevant code in operators implementation.
When the variables are private inside your struct/class then you need to declare operators as friend methods.
You implement any format/separators that you like. When your string include spaces use getline() that takes a string and stream instead of >> because operator>> uses spaces as delimiters by default. Depends on your separators.
It's called serialization. There are many serialization threads on SO.
There are also a nice serialization library included in boost.
http://www.boost.org/doc/libs/1_42_0/libs/serialization/doc/index.html
basically you can do
myFile<<myObject
and
myFile>>myObject
with boost serialization.
If you have:
struct A {
char a[30], b[25], c[15];
int x;
}
then you can write it all just with write(fh, ptr, sizeof(struct a)).
Of course, this isn't portable (because we're not saving the endieness or size of "int," but that may not be an issue for you.
If you have:
struct A {
char *a, *b, *c;
int d;
}
then you're not looking to write the object; you're looking to serialize it. Your best bet is to look in the Boost libraries and use their serialization routines, because it's not an easy problem in languages without reflection.
There's not really a simple way, it's C++ after all, not PHP, or JavaScript.
http://www.parashift.com/c++-faq-lite/serialization.html
Boost also has some library for it: http://www.boost.org/doc/libs/release/libs/serialization ... like Tronic already mentioned :)
The better method is to write each field individually along with the string length.
As an alternative, you can create a char array (or std::vector<char>) and write all the members into the buffer, then write the buffer to the output.
The underlying thorn is that a compiler is allowed to insert padding between members in a class or structure. Use memcpy or std::copy will result in padding bytes written to the output.
Just remember that you need to either write the string lengths and the content or the content followed by some terminating character.
Other people will suggest checking out the Boost Serialization library.
Unfortunately that is generally not quite possible. If your struct only contains plain data (no pointers or complex objects), you can store it as a one chunk, but care must be taken if portability is an issue. Padding, data type size and endianess issues make this problematic.
You can use Boost.Serialization to minimize the amount of code required for proper portable and versionable searialization.
Assuming your goal is as stated, to write out the object with a single call to write() or fwrite() or whatever, you'd first need to copy the string and other object data into a single contiguous block of memory. Then you could write() that block of memory out with a single call. Or you might be able to do a vector-write by calling writev(), if that call is available on your platform.
That said, you probably won't gain much by reducing the number of write calls. Especially if you are using fwrite() or similar already, then the C library is already doing buffering for you, so the cost of multiple small calls is minimal anyway. Don't put yourself through a lot of extra pain and code complexity unless it will actually do some good...