Below is my function to copy file from source to destination
void CopyFile(const std::string& source, const std::string& destination)
{
std::ifstream ifs(source.c_str(), std::ios::in | std::ios::binary | std::ios::ate);
if (!ifs.good())
return;
std::ofstream ofs(destination.c_str(), std::ios::out | std::ios::binary);
const int bufferSize = 4;
char buffer[bufferSize];
std::streamoff streamSize = ifs.tellg();
std::streamoff iteration = streamSize / bufferSize;
int rem = streamSize % bufferSize;
ifs.seekg(std::ios::beg);
while (iteration > 0)
{
ifs.read(buffer, bufferSize);
ofs.write(buffer, bufferSize);
ifs.seekg(bufferSize, std::ios::cur);
ofs.seekp(bufferSize, std::ios::cur);
--iteration;
}
ifs.read(buffer, rem);
ofs.write(buffer, rem);
ifs.close();
ofs.close();
}
other approach where i open file in non binary mode working for me but not this one. what am i missing here?
Related
I am trying to write a char* to a binary file.
This is what I have now.
void Write(char* fileName, char* pData)
{
ofstream binFile (fileName, ios::out | ios::binary);
if (binFile.open())
{
binFile.write((char*)&pData, sizeof(pData));
binFile.close();
}
}
void Read(char* fileName, char* pData)
{
ifstream binFile(fileName, ios::in | ios::binary);
if(binFile.open())
{
binFile.read(char*)&pData, sizeof(pData));
binFile.close
}
}
int main()
{
char* testData = "ABCdEFG"; // not real data
char* getTestData;
char* file = "C:\\testData.dat";
Write(file, testData);
Read(file, getTestData);
}
Test data will be of unknown length. May not always be the same.
When i run the program once, and write and read. I can get back the test data.
But when i stop the program and run it again, this time without writing. Just reading, i cannot get back the test data.
I don't really understand whats happening here.
Can some one explain it to me?
binFile.write((char*)&pData, sizeof(pData));
is wrong. It just writes the value of the pointer. It does not write the data.
You need to use:
binFile.write(pData, strlen(pData));
However, that won't be adequate to read the data back. To be able to read the data back, you'll need to write the size of the string first.
size_t len = strlen(pData);
binFile.write((char*)&len, sizeof(len));
binFile.write(pData, len);
And when reading the data back, you will need to use:
size_t len = 0;
binFile.read(char*)&len, sizeof(len));
binFile.read(pData, len);
and then, null terminate the string.
pData[len] = '\0';
PS
Make sure getTestData is properly initialized before using it to read the data.
char getTestData[100];
will be adequate for your test case.
Update
You can make your program a bit better by using std::string instead of char*. The size of the saved data can be more easily managed when a std::string is used.
void Write(std::string const& fileName, std::string const& data)
{
std::ofstream binFile(fileName, std::ios::out | std::ios::binary);
if (binFile.is_open())
{
size_t len = data.size();
binFile.write((char*)&len, sizeof(len));
binFile.write((char*)&data[0], len);
// No need. The file will be closed when the function returns.
// binFile.close();
}
}
void Read(std::string const& fileName, std::string& data)
{
std::ifstream binFile(fileName, std::ios::in | std::ios::binary);
if(binFile.is_open())
{
size_t len = 0;
binFile.read((char*)&len, sizeof(len));
data.resize(len);
binFile.read((char*)&data[0], len);
}
}
int main()
{
std::string file = "testData.dat";
std::string testData = "ABCdEFG";
Write(file, testData);
std::string getTestData;
Read(file, getTestData);
std::cout << getTestData << std::endl;
}
I was reading sehe's answer for fast text file reading in C++, which looks like this.
static uintmax_t wc(char const *fname)
{
static const auto BUFFER_SIZE = 16*1024;
int fd = open(fname, O_RDONLY);
if(fd == -1)
handle_error("open");
/* Advise the kernel of our access pattern. */
posix_fadvise(fd, 0, 0, 1); // FDADVICE_SEQUENTIAL
char buf[BUFFER_SIZE + 1];
uintmax_t lines = 0;
while(size_t bytes_read = read(fd, buf, BUFFER_SIZE))
{
if(bytes_read == (size_t)-1)
handle_error("read failed");
if (!bytes_read)
break;
for(char *p = buf; (p = (char*) memchr(p, '\n', (buf + bytes_read) - p)); ++p)
++lines;
}
return lines;
}
This is cool, but I was wondering if a similar approach can be taken when we aren't dealing with a character operation like counting newlines, but want to operate on each line of data. Say for instance I had a file of doubles, and already some function parse_line_to_double to use on each line.
12.44243
4242.910
...
That is, how can I read BUFFER_SIZE bytes into my buffer but avoid splitting the last line read? Effectively, can I ask "Give me BUFFER_SIZE or less bytes while ensuring that the last byte read is a newline character (or EOF)"?
Knowing extremely little about low level IO like this, ideas that came to mind were
Can I "back up" fd to the most recent newline between iterations?
Do I have to keep a second buffer holding a copy of the current line being read all the time?
Here is a comparison test. First, lets try the easy way. Just read the file with standard C++ functions:
#include <iostream>
#include <string>
#include <fstream> //std::ifstream
#include <sstream> //std::stringstream
uintmax_t test1(char const *fname)
{
std::ifstream fin(fname);
if(!fin) return 0;
uintmax_t lines = 0;
std::string str;
double value;
while(fin >> value)
{
//std::cout << value << "\n";
lines++;
}
return lines;
}
Next, with std::stringstream this is about 2.5 times faster:
uintmax_t test2(char const *fname)
{
std::ifstream fin(fname);
if(!fin) return 0;
uintmax_t lines = 0;
std::string str;
double value;
std::stringstream ss;
ss << fin.rdbuf();
while(ss >> value)
lines++;
return lines;
}
Next, lets read the whole file in to memory. This will be fine as long as the file is less than 1 GiB or so. Assuming there is a double value on each line, lets extract that value. test3 is more complicated and less flexible, and it's not any faster than test2:
uintmax_t test3(char const *fname)
{
std::ifstream fin(fname, std::ios::binary);
if(!fin) return 0;
fin.seekg(0, std::ios::end);
size_t filesize = (size_t)fin.tellg();
fin.seekg(0);
std::string str(filesize, 0);
fin.read(&str[0], filesize);
double value;
uintmax_t lines = 0;
size_t beg = 0;
size_t i;
size_t len = str.size();
for(i = 0; i < len; i++)
{
if(str[i] == '\n' || i == len - 1)
{
try
{
value = std::stod(str.substr(beg, i - beg));
//std::cout << value << "\n";
beg = i + 1;
lines++;
}
catch(...)
{
}
}
}
return lines;
}
For comparison to the wc function in the question, let's read the whole file in to memory and only count the number of lines. This runs a little faster than wc (as expected), suggesting that there is no need for additional optimizations
uintmax_t test_countlines(char const *fname)
{
std::ifstream fin(fname, std::ios::binary);
if(!fin) return 0;
fin.seekg(0, std::ios::end);
size_t filesize = (size_t)fin.tellg();
fin.seekg(0);
std::string str(filesize, 0);
fin.read(&str[0], filesize);
uintmax_t lines = 0;
for(auto &c : str)
if(c == '\n')
lines++;
return lines;
}
I am trying to write a char* to a binary file.
This is what I have now.
void Write(char* fileName, char* pData)
{
ofstream binFile (fileName, ios::out | ios::binary);
if (binFile.open())
{
binFile.write((char*)&pData, sizeof(pData));
binFile.close();
}
}
void Read(char* fileName, char* pData)
{
ifstream binFile(fileName, ios::in | ios::binary);
if(binFile.open())
{
binFile.read(char*)&pData, sizeof(pData));
binFile.close
}
}
int main()
{
char* testData = "ABCdEFG"; // not real data
char* getTestData;
char* file = "C:\\testData.dat";
Write(file, testData);
Read(file, getTestData);
}
Test data will be of unknown length. May not always be the same.
When i run the program once, and write and read. I can get back the test data.
But when i stop the program and run it again, this time without writing. Just reading, i cannot get back the test data.
I don't really understand whats happening here.
Can some one explain it to me?
binFile.write((char*)&pData, sizeof(pData));
is wrong. It just writes the value of the pointer. It does not write the data.
You need to use:
binFile.write(pData, strlen(pData));
However, that won't be adequate to read the data back. To be able to read the data back, you'll need to write the size of the string first.
size_t len = strlen(pData);
binFile.write((char*)&len, sizeof(len));
binFile.write(pData, len);
And when reading the data back, you will need to use:
size_t len = 0;
binFile.read(char*)&len, sizeof(len));
binFile.read(pData, len);
and then, null terminate the string.
pData[len] = '\0';
PS
Make sure getTestData is properly initialized before using it to read the data.
char getTestData[100];
will be adequate for your test case.
Update
You can make your program a bit better by using std::string instead of char*. The size of the saved data can be more easily managed when a std::string is used.
void Write(std::string const& fileName, std::string const& data)
{
std::ofstream binFile(fileName, std::ios::out | std::ios::binary);
if (binFile.is_open())
{
size_t len = data.size();
binFile.write((char*)&len, sizeof(len));
binFile.write((char*)&data[0], len);
// No need. The file will be closed when the function returns.
// binFile.close();
}
}
void Read(std::string const& fileName, std::string& data)
{
std::ifstream binFile(fileName, std::ios::in | std::ios::binary);
if(binFile.is_open())
{
size_t len = 0;
binFile.read((char*)&len, sizeof(len));
data.resize(len);
binFile.read((char*)&data[0], len);
}
}
int main()
{
std::string file = "testData.dat";
std::string testData = "ABCdEFG";
Write(file, testData);
std::string getTestData;
Read(file, getTestData);
std::cout << getTestData << std::endl;
}
The question in how to read, modify on-fly and save date to the same std::fstream handled file like this:
std::fstream file(input_name.c_str(), std::ios::out | std::ios::in | std::ios::app | std::ifstream::binary);
std::vector<unsigned char> byte;
inputFile.seekg(0, file.end);
long long int length = file.tellg();
file.seekg(0, file.beg);
lldiv_t divresult = lldiv(length, blockInput);
for (long long int a = 0; a != divresult.quot; a++) {
byte.reserve(blockInput / sizeof(unsigned char));
byte.insert(byte.begin(), blockInput / sizeof(unsigned char), 0);
file.read((char*)&byte[0], blockInput);
for (int size_counter = 0; size_counter != blockInput; size_counter++) {
byte[size_counter] = Transform(rounds, byte[size_counter], bytesUsed++);
}
//the same as lower
file.write((char*)&byte[0], blockInput);
byte.clear();
}
if (divresult.rem > 0) {
byte.reserve(divresult.rem / sizeof(unsigned char));
byte.insert(byte.begin(), divresult.rem / sizeof(unsigned char), 0);
file.read((char*)&byte[0], divresult.rem);
for (int size_counter = 0; size_counter != divresult.rem; size_counter++) {
byte[size_counter] = Transform(rounds, byte[size_counter], bytesUsed++);
}
//what should be here?
//perhaps "file.seekg( file.tellg() - bufferSize );" but I'm not sure what to do next..
file.write((char*)&byte[0], divresult.rem);
byte.clear();
}
file.close();
Do you Guys know how to do it correctly?
Nah, the answer is to modify both read and write stream positions by std::istream::seekg and std::istream::seekp in specific places marked by me as here:
std::fstream file(input.c_str(), std::ios::in | std::ios::out | std::ifstream::binary);
std::vector<unsigned char> byte;
file.seekg(0, file.end);
long long int length = file.tellg();
file.seekg(0, file.beg);
file.seekp(0, file.beg);
lldiv_t divresult = lldiv(length, blockInput);
for (long long int a = 0; a != divresult.quot; a++) {
std::vector<unsigned char> byte;
byte.reserve(blockInput / sizeof(unsigned char));
byte.insert(byte.begin(), blockInput / sizeof(unsigned char), 0);
//HERE
file.seekp(a*blockInput, std::ios_base::beg);
file.read((char*)&byte[0], blockInput);
for (int size_counter = 0; size_counter != blockInput; size_counter++) {
byte[size_counter] = Transform(rounds, byte[size_counter]);
}
//HERE
file.seekg(a*blockInput, std::ios_base::beg);
file.write((char*)&byte[0], blockInput);
byte.clear();
}
if (divresult.rem > 0) {
byte.reserve(divresult.rem / sizeof(unsigned char));
byte.insert(byte.begin(), divresult.rem / sizeof(unsigned char), 0);
//HERE
file.seekp(divresult.quot*blockInput, std::ios_base::beg);
file.read((char*)&byte[0], divresult.rem);
for (int size_counter = 0; size_counter != divresult.rem; size_counter++) {
byte[size_counter] = Transform(rounds, byte[size_counter]);
}
//AAAND HERE
file.seekg(divresult.quot*blockInput, std::ios_base::beg);
file.write((char*)&byte[0], divresult.rem);
byte.clear();
}
file.close();
Seem to be easy but, it was hard to find out what the problem actually is. I hope somebody will be able to use it somewhere :)
Several topics (see Using C++ filestreams (fstream), how can you determine the size of a file? and C++: Getting incorrect file size) on how to measure a file size compute the difference between the beginning and the end of the file like that :
std::streampos fileSize( const char* filePath ){
std::streampos fsize = 0;
std::ifstream file( filePath, std::ios::binary );
fsize = file.tellg();
file.seekg( 0, std::ios::end );
fsize = file.tellg() - fsize;
file.close();
return fsize;
}
But instead of opening the file at the beginning, we can open it at the end and just take a measure, like that :
std::streampos fileSize( const char* filePath ){
std::ifstream file( filePath, std::ios::ate | std::ios::binary );
std::streampos fsize = file.tellg();
file.close();
return fsize;
}
Will it work ? And if not why ?
It should work just fine. The C++ standard says about std::ios::ate
ate - open and seek to end immediately after opening
There's no reason it would fail when a manual open-then-seek would succeed. And tellg is the same in either case.