I wanted to shrink the size of a large text file with float values into a binary .dat file, so I used (in c++):
// the text stream
std::ifstream fin(sourceFile);
// the binary output stream
std::ofstream out(destinationFile, std::ios::binary);
float val;
while(!fin.eof())
{
fin >> val;
out.write((char *)&val,sizeof(float));
}
fin.close();
out.close();
Then, I wanted to read all the float values from the rpeviously created binary file into a array of float values.
But when I try to read from this file I get an exception at the last line of code (the reading process):
// test read
std::ifstream fstream(destinationFile, std::ios::binary);
__int64 fileSize = 0;
struct __stat64 fileStat;
if(0 == _tstat64(destinationFile, &fileStat))
{
fileSize = fileStat.st_size;
}
//get the number of float tokens in the file
size_t tokensCount = fileSize / sizeof(float);
float* pBuff = new float[tokensCount];
fstream.read((char*)&pBuff, tokensCount * sizeof(float));
What am I doing wrong?
float* pBuff = new float[tokensCount];
fstream.read((char*)&pBuff, tokensCount * sizeof(float));
You are reading into the pBuff variable, not the buffer it points to. You mean:
fstream.read((char*)pBuff, tokensCount * sizeof(float));
Note that this:
while(!fin.eof())
{
fin >> val;
out.write((char *)&val,sizeof(float));
}
is not the correct way to read a file - it will read a garbage value at the end. You should almost never use the eof() function and you should ALWAYS check that a file read worked. Correct code is:
while( fin >> val )
{
out.write((char *)&val,sizeof(float));
}
Magnus' answer is correct and should solve your problem. I will only add that you wouldn't have had a problem in the first place if you had done as the gurus say and not used an evil C-style cast. If you change your last line to this:
fstream.read(static_cast<char*>(&pBuff), tokensCount * sizeof(float));
Then your program would have failed to compile and the error message would have led you to the solution.
EDIT: my solution does not work if pBuff is a pointer to any type other than char. So it's no use in the OP's case.
Related
In the following code the read method doesn't seem to fill the given buffer:
ifstream pkcs7_file(file_name, std::ios::binary);
if ( pkcs7_file.fail() )
{
std::cout << "File failed before reading!\n";
}
pkcs7_file.seekg(0, pkcs7_file.end);
size_t len = pkcs7_file.tellg();
char * buffer = new char[len];
pkcs7_file.read(buffer, len);
pkcs7_file.close();
When debugging with VS 2012 and printing, the Len variable is as expected (and not zero) but the buffer doesn't change after the read function - it remains with the same value from before the read.
What am I doing wrong?
You seek to end-of-file, and then try to read. Of course it fails - the file is positioned at EOF, there's no data to read.
I need to read all blocks of one large file(about 10GB) sequentially, the file contains many floats with a few strings, like this(each item splited by '\n'):
6.292611
-1.078219E-266
-2.305673E+065
sod;eiwo
4.899747e-237
1.673940e+089
-4.515213
I read MAX_NUM_PER_FILE items each time and process them and write to another file, but i don't know when the ifstream is ended.
Here is my code:
ifstream file_input(path_input); //my file is a text file, but i tried both text and binary mode, both failed.
if(file_input)
{
file_input.seekg(0,file_input.end);
unsigned long long length = file_input.tellg(); //get file size
file_input.seekg(0,file_input.beg);
char * buffer = new char [MAX_NUM_PER_FILE+MAX_NUM_PER_LINE];
int i=1,j;
char c,tmp[3];
while(file_input.tellg()<length)
{
file_input.read(buffer,MAX_NUM_PER_FILE);
j=MAX_NUM_PER_FILE;
while(file_input.get(c)&&c!='\n')
buffer[j++]=c; //get a complete item
//process with buffer...
itoa(i++,tmp,10); //int2char
string out_name="out"+string(tmp)+".txt";
ofstream file_output(out_name);
file_output.write(buffer,j);
file_output.close();
}
file_input.close();
delete[] buffer;
}
My code goes wrong, length is bigger than real file size. I have tried file_input.good() or !file_input.eof(), they didn't work, getline(file_input,s) is good, but it is much slower than read, i want read, but i don't know how to check whether ifstream is end-of-file.
I do my work in WINDOWS 7 with VS2010.
I have searched, but there are not any answer about it, How to open a file using ifstream and keep reading it until the end this link can't answer my question.
Update, Problem solved
Hi everyone, I have figured it out that it's my fault. Both while(file_input.tellg()<length) and while(file_input.peek()!=EOF) work fine! while(file_input.peek()!=EOF) is recommended.
The extra items written after the end-of-file is the left items in buffer written in the last time.
Here is the correct code:
ifstream file_input(path_input);
if(file_input)
{
//file_input.seekg(0,file_input.end);
//unsigned long long length = file_input.tellg(); //get file size
//file_input.seekg(0,file_input.beg);
char * buffer = new char [MAX_NUM_PER_FILE+MAX_NUM_PER_LINE];
int i=1,j;
char c,tmp[3];
while(file_input.peek()!=EOF)
{
memset(buffer,0,sizeof(char)*(MAX_NUM_PER_FILE+MAX_NUM_PER_LINE)); //clear first!
file_input.read(buffer,MAX_NUM_PER_FILE);
j=MAX_NUM_PER_FILE;
while(file_input.get(c)&&c!='\n')
buffer[j++]=c;
itoa(i++,tmp,10);//int2char
string out_name="out"+string(tmp)+".txt";
ofstream file_output(out_name);
file_output.write(buffer,strlen(buffer)); //use the correct buffer size instead of j
file_output.close();
}
file_input.close();
delete[] buffer;
}
while( file_input.peek() != EOF )
{
// code
}
Basically peek() will read the next char without extracting it.
So you can simply compare it to EOF.
I have to write some data to a text file, and at the end of each output I have to append a NULL terminating character '\0'. Currently this is what I have come up so far. It works well for some inputs, however for some it sometimes write the whole text file with garbage value. I there a better way to do this?? In my program I have to write some data, store its location on file and use that for some operations. the next write operation starts at address = address + 500;
long int address = get_address();
void write_to_file()
{
fstream pFILE ("my file.txt");
char * buffer = new char [500];
cin.getline(buffer,500);
pFILE.seekp(address);
pFILE << buffer;
pFILE.seekp(address + strlen(buffer));
pFILE << '\0';
address += 500;
}
To write a '\0' to file:
fstream output_file("output_file.txt", ios::binary);
output_file.put('\0');
The ios::binary prevents the compiler or OS from translating the '\0'.
I want to copy a file by reading blocks of data, sending it and than put it back together again. Sending is not part of the problem, so I left it out in the code. It should work with any type of file and arbitrary piece_lengths.
This is just a pre-stage. In the end data block should not be chosen sequentially but at random. There could be some time between receiving another block of data.
I know the example just makes sense if size % piece_length != 0.
I'm getting crashed files of the same size as the original file at the other end.
Does anyone see the problem?
int main ()
{
string file = "path/test.txt"
string file2 = "path2/test.txt";
std::ifstream infile (file.c_str() ,std::ifstream::binary);
//get size of file
infile.seekg (0,infile.end);
long size = infile.tellg();
infile.seekg (0);
size_t piece_length = 5;
for (int i = 0; i < ((size / piece_length) + 1); i++)
{
if ( i != (size / piece_length))
{
std::ifstream infile (file.c_str() ,std::ifstream::binary);
infile.seekg((i * piece_length) , infile.beg);
char* buffer = new char[piece_length];
infile.read(buffer, piece_length);
infile.close();
std::ofstream outfile (file2.c_str() ,std::ofstream::binary);
outfile.seekp((i * piece_length), outfile.beg);
outfile.write(buffer, piece_length);
outfile.close();
}
else
{
std::ifstream infile (file.c_str() ,std::ifstream::binary);
infile.seekg((i * piece_length) , infile.beg);
char* buffer = new char[size % piece_length];
infile.read(buffer, size % piece_length);
infile.close();
std::ofstream outfile (file2.c_str() ,std::ofstream::binary);
outfile.seekp((i * piece_length), outfile.beg);
outfile.write(buffer, size % piece_length);
outfile.close();
}
}
return 0;
}
To answer your specific question, you need to open outfile with ios::in | ios::out in the flags, otherwise it defaults to write-only mode and destroys what was already in the file. See this answer for more details: Write to the middle of an existing binary file c++
You may want to consider the following though:
If you are just writing parts to the end of the file, just use ios::app (append). Don't even need to seek.
You don't need to keep reopening infile or even outfile, just reuse them.
You can also reuse buffer. Please remember to delete them, or better yet use a std::vector.
This is basically the part of the code that i used to store the entire file, and works well ... but when i tryed to store a integer bigger than 120 or something like that the program writes seems like a bunch of trash and not the integer that i want. Any tips ? I am an college student and dont have a clue whats happening.
int* temp
temp = (int*) malloc (sizeof(int));
*temp = atoi( it->valor[i].c_str() );
//Writes the integer in 4 bytes
fwrite(temp, sizeof (int), 1, arq);
if( ferror(arq) ){
printf("\n\n Error \n\n");
exit(1);
}
free(temp);
I've already checked the atoi part and it really returns the number that I want to write.
I changed and added some code and it works fine:
#include <iostream>
using namespace std;
int main()
{
int* temp;
FILE *file;
file = fopen("file.bin" , "rb+"); // Opening the file using rb+ for writing
// and reading binary data
temp = (int*) malloc (sizeof(int));
*temp = atoi( "1013" ); // replace "1013" with your string
//Writes the integer in 4 bytes
fwrite(temp, sizeof (int), 1, file);
if( ferror(file) ){
printf("\n\n Error \n\n");
exit(1);
}
free(temp);
}
Make sure you are opening the file with the correct parameters, and that the string you give to atoi(str) is correct.
I checked the binary file using hex editor, after inputting the number 1013.
int i = atoi("123");
std::ofstream file("filename", std::ios::bin);
file.write(reinterpret_cast<char*>(&i), sizeof(i));
Do not use pointers here.
Never use malloc / free in C++.
Use C++ file streams, not C streams.