Copying QFile contents to another QFile, what's the optimal way? - c++

Question

I need to copy a QFile to another QFile in chunks, so I can't use QFile::copy. Here's the most primitive implementation:
bool CFile::copyChunk(int64_t chunkSize, const QString &destFolder)
{
if (!_thisFile.isOpen())
{
// Initializing - opening files
_thisFile.setFileName(_absoluteFilePath);
if (!_thisFile.open(QFile::ReadOnly))
return false;
_destFile.setFileName(destFolder + _thisFileName);
if (!_destFile.open(QFile::WriteOnly))
return false;
}
if (chunkSize < (_thisFile.size() - _thisFile.pos()))
{
QByteArray data (chunkSize, 0);
_thisFile.read(data.data(), chunkSize);
return _destFile.write(data) == chunkSize;
}
}
It's not clear from this fragment, but I only intend to copy a binary file as a whole into another location, just in chunks so I can provide progress callbacks and cancellation facility for large files.
Another idea is to use memory mapping. Should I? If so, then should I only map source file and still use _destFile.write, or should I map both and use memcpy?
I guess this question isn't really tied to Qt, I think the answer should be general to any file I/O API that supports memory mapping.

Answer 1

Ok, ok, if it must be a memory mapping solution. Here is one:
QFile source("/tmp/bla1.bin");
source.open(QIODevice::ReadOnly);
QFile destination("/tmp/bla2.bin");
destination.open(QIODevice::ReadWrite);
destination.resize(source.size());
uchar *data = destination.map(0,destination.size());
if(!data){
qDebug() << "Cannot map";
exit(-1);
}
QByteArray buffer;
int chunksize = 200;
int var = 0;
do{
var = source.read((char *)(data), chunksize);
data += var;
}while(var > 0);
destination.unmap(data);
destination.close();
This maps only the destination file into memory. I doubt it will make much of a difference to map the source file also. But this is something for concrete measurements, not assumptions.
Another questions is whether you can map your whole file into memory at once. Constantly unmapping and remapping will certainly cost performance. And even if you use Qt. Functions like memory mapping have the tendency to act disturbingly different on different platforms, e.g. the maximum file size you map in to memory might be different.

Answer 2

What the optimal method is, lies always a bit in the eye of the beholder. Here is at least one working shorter method:
QFile source("/tmp/bla1.bin");
source.open(QIODevice::ReadOnly);
QFile destination("/tmp/bla2.bin");
destination.open(QIODevice::WriteOnly);
QByteArray buffer;
int chunksize = 200; // Whatever chunk size you like
while(!(buffer = source.read(chunksize)).isEmpty()){
destination.write(buffer);
}
destination.close();
source.close();
And memory mapping... I try to stay away from things like that. I am never too sure how platform independent they are.

Answer 3

Use this QFile::map() method:
QFile fs("Sourcefile.bin");
fs.open(QFile::ReadOnly);
QFile fd("Destinationfile.bin");
fd.open(QFile::WriteOnly);
fd.write((char*) fs.map(0, fs.size()), fs.size()); //Copies all data
fd.close();
fs.close();