I'am having troubles with fstream while reading large binary files(2 GigaBytes). Using this test code I create fstream and then read until EOF. Then clear flags and reset position to the beginning of the file. Then I read it again. Problem is, that the second read(second while cycle) is always significantly slower.
I need to get this solved for Embarcadero RAD Studio XE7. Same beheaviour of slower second read can be replicated in MS Visual 2010. First read is done in HDD maximum speed (about 140 MB/s), second is always done at quarter of it (35 MB/s).
Oddly, when I use g++ 4.9.2 on my Debian, both first and second readings are done with the same time as I would expected.
#include <stdio.h>
#include <cstdlib>
#include <fstream>
#include <string>
char buffer[400000];
int main()
{
std::fstream stream;
std::string filename = "largeFile.bin";
stream.open(filename, std::ios_base::in | std::ios_base::binary);
while(stream.good()){
stream.read(buffer, 400000);
}
printf("first read complete");
stream.clear();
stream.seekg(std::ios::beg);
while(stream.good()){
stream.read(buffer, 400000);
}
printf("second read complete");
stream.close();
return 0;
}
I omitted time measure of both while cycles, since it is not significant for this problem.
Am I doing something wrong when reading multiple times from start to EOF, on once opened file?
Related
I am having a head-scratcher and need a human more intelligent than I to tell me where I am wrong/ what is going on.
All I am trying to do is open a file, read the number, add one to the number, truncate the file with my new number, and be done. The purpose is so that I can keep a memory even after my (other larger) program is shutdown. Each time I run the function it does weird stuff
#include <iostream>
#include <fstream>
using namespace std;
int getcount(){
int number;
ifstream indata;
ofstream otdata;
indata.open("count.txt");
indata>>number;
number++;
otdata.open("count.txt", ios::trunc);
otdata.close();
return number;
}
int main()
{
int i;
i=getcount();
cout<<i<<endl;
}
my current output looks like this
32575
32655
65536
65536
65536
32622
Is there a better way to do this? Where should I look? Did I typo? I have done stuff just like this before and have read the documentation, my college books, etc, cant find anything revealing. Thank you in advance.
Does count.txt exist? If it does, it looks like you're not storing the number after you've opened it (nothing that writes back the number between otdata.open and otdata.close). Also, you should close indata after you're finished reading it.
I'm making a C++ program to generate a million random numbers (I've generated them as just cout output once so I have the processing power) and I want to write them into a file.
#include <iostream>
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <fstream>
using namespace std;
int main() {
srand(time(NULL));
int value;
ofstream file ("numbers.txt");
file.open ("numbers.txt", ios::out | ios::app );
int i = 0;
while (i < 1000000)
{
value = rand();
file << value;
}
file.close();
}
This is my current code, and I get no errors, but when I run it I see an empty txt file in my file explorer.
Can anyone tell me what's the problem here?
I tried it for only 100 numbers, and I got a blank text file
Don't reopen the same file:
...
ofstream file ("numbers.txt");
file.open ("numbers.txt", ios::out | ios::app ); // Remove this line
int i = 0;
...
That will make it work. However, the program won't ever stop, since you forgot to increment i too!
As said by other users in comments, you have an infinite loop: incrementing your loop control variable solves this problem. Then is not necessary to open and close the file explicitly if you use the standard library.
I would add that generally speaking rand() is not the function you may want to use, because it generates pseudo-random numbers, and even if it is ok for your application you may want to give a seed to it, throught the function srand().
If you want more from you program, have a look a this reference page.
#include <iostream>
#include <fstream>
#include <conio.h>
using namespace std;
int main()
{
char r;
fstream file1("text.txt", ios::in |ios::binary);
fstream file2("text.txt", ios::out | ios::binary);
r='r';
for(int i=0; i<100; i++)
file2.write((char*)r, sizeof(char));
while(!file1.eof())
{
file1.read((char*)r, sizeof(char));
cout<<r<<"\n";
}
file1.close();
file2.close();
getch();
}
when I run this in vc++ 2010, I get the following error during run time:
Unhandled exception at 0x55361f68 (msvcp100d.dll) in file io.exe: 0xC0000005: Access violation reading location 0x00000072.
what could be causing this error? this happens while reading the line :
file2.write((char*)r, sizeof(char));
did I make any mistake? If yes please point it out for me (thanks in advance).
Update: I am still not getting my expected output (after correcting (char*)r to (char*)&r). the output that I get is just: r. shouldn't I expect 100 characters to be displayed starting from 'r'? If not, please tell me why and thanks in advance.
You need
file1.read((char*)&r, sizeof(char));
or
file1.read(&r, sizeof(char));
In addition to the other answer, there's also another problem your code has.
Streams perform buffered I/O by default; when writing into file1, the contents that you've written probably haven't been outputted to the actual file yet. The contents are actually stored in a temporary buffer for efficiency. Writing to the actual file is an operation reserved for an explicit flush(), when close() is called, or when the file stream goes out of scope and is destructed.
The problem in your code is that directly after writing to the file stream, you perform input without determining whether that output data was written to the actual file. This can cause Undefined Behavior if you assume that the data was read successfully from the input file to the variable.
File streams that depend on each other should be synchronized. Namely, when a file stream is trying to read from the same file that you have written to, then the output file stream must be flushed. This can be facilitated by "tying" the streams together, this is done using tie():
file1.tie(&file2);
When file1 performs input, file2 will then be flushed, forcing the data in its buffer to be written the file.
Another problem you have is that you don't check if the file streams were constructed correctly, or that you have successfully read from file1. You can use if() statements to do this:
std::fstream file1("text.txt", std::ios_base::in | std::ios_base::binary);
std::fstream file2("text.txt", std::ios_base::out | std::ios_base::binary);
char r('r');
if (file1 && file2)
{
file1.tie(&file2);
for (int i = 0; i < 100; ++i)
file2.write(&r, sizeof(char));
while (file1.read(&r, sizeof(char))) {
std::cout << r << std::endl;
}
}
You started reading from a file immediately after writing on that file and without closing the write file stream. Until you close the write file stream it will not commit the writings. So there is a change of getting access violation as it holds the control.
Try following code
#include <iostream>
#include <fstream>
#include <conio.h>
using namespace std;
int main()
{
char *r="r";
fstream file2("text.txt", ios::out | ios::binary);
for(int i=0; i<100; i++)
file2.write((char*)r, sizeof(char));
file2.close();
fstream file1("text.txt", ios::in |ios::binary);
while(!file1.eof())
{
char rr;
file1.read(&rr, sizeof(char));
cout<<rr<<"\n";
}
file1.close();
getch();
}
You have tried to cast a single char to char * and also tried to read using fread without passing r's address. That's why the problem is occurring. Please carefully see my code above, it will fix your issues.
I'm trying to write simple c++ code to read and write a file.
The problem is my output file is smaller than the original file, and I'm stuck finding the cause.
I have a image with 6.6 kb and my output image is about 6.4 kb
#include <iostream>
#include <fstream>
using namespace std;
ofstream myOutpue;
ifstream mySource;
int main()
{
mySource.open("im1.jpg", ios_base::binary);
myOutpue.open("im2.jpg", ios_base::out);
char buffer;
if (mySource.is_open())
{
while (!mySource.eof())
{
mySource >> buffer;
myOutpue << buffer;
}
}
mySource.close();
myOutpue.close();
return 1;
}
Why not just:
#include <fstream>
int main()
{
std::ifstream mySource("im1.jpg", std::ios::binary);
std::ofstream myOutpue("im2.jpg", std::ios::binary);
myOutpue << mySource.rdbuf();
}
Or, less chattily:
int main()
{
std::ofstream("im2.jpg", std::ios::binary)
<< std::ifstream("im1.jpg", std::ios::binary).rdbuf();
}
Two things: You forget to open the output in binary mode, and you can't use the input/output operator >> and << for binary data, except if you use the output operator to write the input-streams basic_streambuf (which you can get using rdbuf).
For input use read and for output use write.
There are 3 problems in your code:
1- You have not opened your output file in Binary.
2- Your code return "1", normally you should return "0", if something went wrong then return an error code.
3- You should use "manipulators" and make c++ not to avoid whitespaces, so in order to read from file instead of:
mySource >> buffer;
you should use:
mySource >> std:noskipws >> buffer;
Well, its just because of padding at the end of the image. eof of any file do not include the padded bytes added at the end of file.
Try this
take img1.jpg contains 20 space charecter at the end not visible here (uegfuyregwfyugwrerycgerfcg6ygerbucykgeugcrgfrgeyf ) and run your program (do not include parenthesis in the file, these are used to show the data content)
you will see img2.jpg contains (uegfuyregwfyugwrerycgerfcg6ygerbucykgeugcrgfrgeyf)
So, its better option to read the file byte by byte using the filesize which you can get using stat, and run for loop till filesize. Hope this should resolve your problem you mentioned above
Solved! thanks all of you very much. My day has been made!(well morning, its 4am)
I'm trying to write a program in C++ that opens a .dat file in binary and replaces the first 1840 hex characters with that of another .dat file, while leaving the remaining hex values of the first .dat file the same. I have spent about 12 hours on this today and have had little success. I am a beginner programmer, I have taken one semester worth of c++ courses and we did not get to streams.
(it opens a file and everything, but deletes every thing after the new values have been added)
#include <iostream>
#include <iomanip>
#include <fstream>
#include <string>
#include <cmath>
#include <cstring>
using namespace std;
int main (){
string filename;
long size;
char* memblock;
cout << " Enter a file to be modded by Mod.dat ";
cin >> filename;
ofstream infile ( filename ,std::ofstream::binary);
//filename: the file that will be opened and changed)
ifstream modFile ("Mod.dat", ifstream::binary);
// (mod.dat is the file that i get the first 1840 hex values from)
modFile.seekg (0,modFile.end);
size = modFile.tellg();
memblock = new char [size];
modFile.seekg (0, ios::beg);
modFile.read (memblock, size);
infile.write(memblock, 1840);
modFile.close();
infile.close();
cout << endl;
return 0;
}
Any help would be greatly appreciated, I hope there is some simple way to do this.
Solved! thanks all of you very much. My day has been made!(well morning, its 4am)
Edit:
You can modidy your file in place with something like :
std::fstream s(my_file_path, std::ios_base::binary);
s.seekp(position_of_data_to_overwrite, std::ios_base::beg);
s.write(my_data, size_of_data_to_overwrite);
std::fstream will not truncate your input file as std::ofstream does.
The other solution is to not use the same file for reading and writing. Use three files :
One for the output file.
One for the First input file.
One for the second input file.
fstream infile ( filename ,std::ofstream::binary); does not keeps the contents of the original file. Everything you write will erase the contents of the file.
Thus, you should:
open the output file
open the "Mod" file, read the first 1840 bytes from the first file, write them into the output file.
open the "main input file" file, move the cursor to 1840, read the remaining data and write it to the output file.
Depending on the "main input file" size, you may want to buffer you read/write operation.
My preferred fix, although Matthieu Rouget's fix does indeed work, is to just add ofstreeam::in to the opening of the input file:
ofstream infile ( filename.c_str(), std::ofstream::binary | ofstream::in);
(I had to use c_str() in my build, as glibc in my version doesn't take std::string as input).
I tested this on my local system (it took a while to realize that mod.dat is actually "Mod.dat"!)
It is probably a good idea to also check that the files actually opened, so something like this after ofstream infile line:
if (!infile)
{
cout << "Couldn't open " << filename << endl;
}
and similar for the modfile line.
And since you go through the effort of figuring out what the first part of the modfile size is, I would suggest that you also USE that for the writing of the file.