I have a custom file with mixed data. At the end of the file there's an entire image which I want to retrieve.
The problem is that, when I 'extract' it and paste it into an image file, rdbuf() leaves me some annoying CR LF characters instead of just the LF ones in the original.
I have already opened both streams in binary mode.
using namespace std;
ifstream i(f, ios::in | ios::binary);
bool found = false; // Found image name
string s; // String to read to
string n = ""; // Image name to retrieve
while (!found) {
getline(i, s);
// Check if it's the name line
if (s[0]=='-' && s[1]=='|' && s[2]=='-') {
found = true;
// Loop through name X: -|-XXXX-|-
// 0123456789
// Length: 10 3 6
for (unsigned int j=3; j<s.length()-4; j++)
n = n + s[j];
}
}
ofstream o(n.c_str(), ios::out | ios::binary);
o << i.rdbuf();
I made some research and find out that << operator treats input as text and so adjusts \n to \r\n on Windows.
A way to prevent this using the write method instead of <<.
You can do it like this (replacing your last line of code):
// get pointer to associated buffer object
std::filebuf* pbuf = i.rdbuf();
// next operations will calculate file size
// get current position
const std::size_t current = i.tellg();
// move to the end of file
i.seekg(0, i.end);
// get size of file (current position of the end)
std::size_t size = i.tellg();
// get size of remaining data (removing the current position from the size)
size -= current;
// move back to where we were
i.seekg(current, i.beg);
// allocate memory to contain image data
char* buffer=new char[size];
// get image data
pbuf->sgetn (buffer,size);
// close input stream
i.close();
// write buffer to output
o.write(buffer,size);
// free memory
delete[] buffer;
Solved. The problem were made during the ofstream operation to save the file before opening. Because the file were saved as text (with CR LF), it was opening as text as well.
Related
I need to read all blocks of one large file(about 10GB) sequentially, the file contains many floats with a few strings, like this(each item splited by '\n'):
6.292611
-1.078219E-266
-2.305673E+065
sod;eiwo
4.899747e-237
1.673940e+089
-4.515213
I read MAX_NUM_PER_FILE items each time and process them and write to another file, but i don't know when the ifstream is ended.
Here is my code:
ifstream file_input(path_input); //my file is a text file, but i tried both text and binary mode, both failed.
if(file_input)
{
file_input.seekg(0,file_input.end);
unsigned long long length = file_input.tellg(); //get file size
file_input.seekg(0,file_input.beg);
char * buffer = new char [MAX_NUM_PER_FILE+MAX_NUM_PER_LINE];
int i=1,j;
char c,tmp[3];
while(file_input.tellg()<length)
{
file_input.read(buffer,MAX_NUM_PER_FILE);
j=MAX_NUM_PER_FILE;
while(file_input.get(c)&&c!='\n')
buffer[j++]=c; //get a complete item
//process with buffer...
itoa(i++,tmp,10); //int2char
string out_name="out"+string(tmp)+".txt";
ofstream file_output(out_name);
file_output.write(buffer,j);
file_output.close();
}
file_input.close();
delete[] buffer;
}
My code goes wrong, length is bigger than real file size. I have tried file_input.good() or !file_input.eof(), they didn't work, getline(file_input,s) is good, but it is much slower than read, i want read, but i don't know how to check whether ifstream is end-of-file.
I do my work in WINDOWS 7 with VS2010.
I have searched, but there are not any answer about it, How to open a file using ifstream and keep reading it until the end this link can't answer my question.
Update, Problem solved
Hi everyone, I have figured it out that it's my fault. Both while(file_input.tellg()<length) and while(file_input.peek()!=EOF) work fine! while(file_input.peek()!=EOF) is recommended.
The extra items written after the end-of-file is the left items in buffer written in the last time.
Here is the correct code:
ifstream file_input(path_input);
if(file_input)
{
//file_input.seekg(0,file_input.end);
//unsigned long long length = file_input.tellg(); //get file size
//file_input.seekg(0,file_input.beg);
char * buffer = new char [MAX_NUM_PER_FILE+MAX_NUM_PER_LINE];
int i=1,j;
char c,tmp[3];
while(file_input.peek()!=EOF)
{
memset(buffer,0,sizeof(char)*(MAX_NUM_PER_FILE+MAX_NUM_PER_LINE)); //clear first!
file_input.read(buffer,MAX_NUM_PER_FILE);
j=MAX_NUM_PER_FILE;
while(file_input.get(c)&&c!='\n')
buffer[j++]=c;
itoa(i++,tmp,10);//int2char
string out_name="out"+string(tmp)+".txt";
ofstream file_output(out_name);
file_output.write(buffer,strlen(buffer)); //use the correct buffer size instead of j
file_output.close();
}
file_input.close();
delete[] buffer;
}
while( file_input.peek() != EOF )
{
// code
}
Basically peek() will read the next char without extracting it.
So you can simply compare it to EOF.
I want to copy a file by reading blocks of data, sending it and than put it back together again. Sending is not part of the problem, so I left it out in the code. It should work with any type of file and arbitrary piece_lengths.
This is just a pre-stage. In the end data block should not be chosen sequentially but at random. There could be some time between receiving another block of data.
I know the example just makes sense if size % piece_length != 0.
I'm getting crashed files of the same size as the original file at the other end.
Does anyone see the problem?
int main ()
{
string file = "path/test.txt"
string file2 = "path2/test.txt";
std::ifstream infile (file.c_str() ,std::ifstream::binary);
//get size of file
infile.seekg (0,infile.end);
long size = infile.tellg();
infile.seekg (0);
size_t piece_length = 5;
for (int i = 0; i < ((size / piece_length) + 1); i++)
{
if ( i != (size / piece_length))
{
std::ifstream infile (file.c_str() ,std::ifstream::binary);
infile.seekg((i * piece_length) , infile.beg);
char* buffer = new char[piece_length];
infile.read(buffer, piece_length);
infile.close();
std::ofstream outfile (file2.c_str() ,std::ofstream::binary);
outfile.seekp((i * piece_length), outfile.beg);
outfile.write(buffer, piece_length);
outfile.close();
}
else
{
std::ifstream infile (file.c_str() ,std::ifstream::binary);
infile.seekg((i * piece_length) , infile.beg);
char* buffer = new char[size % piece_length];
infile.read(buffer, size % piece_length);
infile.close();
std::ofstream outfile (file2.c_str() ,std::ofstream::binary);
outfile.seekp((i * piece_length), outfile.beg);
outfile.write(buffer, size % piece_length);
outfile.close();
}
}
return 0;
}
To answer your specific question, you need to open outfile with ios::in | ios::out in the flags, otherwise it defaults to write-only mode and destroys what was already in the file. See this answer for more details: Write to the middle of an existing binary file c++
You may want to consider the following though:
If you are just writing parts to the end of the file, just use ios::app (append). Don't even need to seek.
You don't need to keep reopening infile or even outfile, just reuse them.
You can also reuse buffer. Please remember to delete them, or better yet use a std::vector.
I am trying to retrieve some data from a binary file to put them in a linked list, here's my code to write to the file:
void Pila::memorizzafile()
{
int contatore = 0;
puntarec temp = puntatesta;
ofstream miofile;
miofile.open("data.dat" , ios::binary | ios::out);
if(!miofile) cerr << "errore";
else
{
while(temp)
{
temp->elem.writes(miofile);
contatore++;
temp = temp->next;
}
//I go back at the beginning of the file to write how many elements I have
miofile.seekp(0, ios::beg);
miofile.write((const char *)&contatore , sizeof(int));
miofile.close();
}
}
And the function writes:
void Fiche::writes(ofstream &miofile)
{
//Valore.
miofile.write((const char *)&Valore,sizeof(int));
//Materiale, I write the dimension of the string.
int buff = strlen(Materiale);
miofile.write((const char *)&buff,sizeof(int));
//Writing the string
miofile.write(Materiale,buff*sizeof(char));
//Dimension of Forma
buff = strlen(Forma);
miofile.write((const char*)&buff,sizeof(int));
//The string itself
miofile.write(Forma,buff*sizeof(char));
//Dimension of Colore.
buff = strlen(Colore);
miofile.write((const char*)&buff,sizeof(int));
//The string
miofile.write(Colore,buff*sizeof(char));
}
Now for the reading part, I am trying to make a constructor which should be able to read directly from the file, here it is:
Pila::Pila(char * nomefile)
{
puntatesta = 0;
int contatore = 0;
ifstream miofile;
miofile.open(nomefile , ios::binary | ios::in);
if(!miofile) cerr << "errore";
else
{
//I read how many records are stored in the file
miofile.read((char*)&contatore,sizeof(int));
Fish temp;
for(int i = 0; i < contatore; i++)
{
temp.reads(miofile);
push(temp);
}
miofile.close();
}
}
And the reading function:
void Fiche::reads(ifstream &miofile)
{
//I read the Valore
miofile.read((char*)&Valore,sizeof(int));
//I create a temporary char *
char * buffer;
int dim = 0;
//I read how long will be the string
miofile.read((char*)&dim,sizeof(int));
buffer = new char[dim];
miofile.read(buffer,dim);
//I use the set function I created to copy the buffer to the actual member char*
setMateriale(buffer);
delete [] buffer;
//Now it pretty much repeats itself for the other stuff
miofile.read((char*)&dim,sizeof(int));
buffer = new char[dim];
miofile.read(buffer,dim);
setForma(buffer);
delete [] buffer;
//And again.
miofile.read((char*)&dim,sizeof(int));
buffer = new char[dim];
miofile.read(buffer,dim);
setColore(buffer);
delete [] buffer;
}
The code doesn't give me any error, but on the screen I read random characters and not even remotely close to what I wrote on my file. Anyone could help me out, please?
EDIT:
As requested here's an example of input&output:
Fiche A("boh" , 4 , "no" , "gaia");
Fiche B("Marasco" , 3 , "boh" , "nonnt");
Fiche C("Valori" , 6 , "asd" , "hey");
Fiche D("TipO" , 7 , "lol" , "nonloso");
Pila pila;
pila.push(A);
pila.push(B);
pila.push(C);
pila.push(D);
pila.stampa();
pila.memorizzafile();
And:
Pila pila("data.dat");
pila.stampa();
This is probably your error:
//I go back at the beginning of the file to write how many elements I have
miofile.seekp(0, ios::beg);
miofile.write((const char *)&contatore , sizeof(int));
miofile.close();
By seeking to the beginning and then writing. You are overwriting part of the first object.
I think your best bet is to run through the list and count the elements first. Write this then proceed to write all the elements. It will probably be faster anyway (but you can time it to make sure).
I think you are using way to many C structures to hold things.
Also I would advice against a binary format unless you are saving huge amounts of information. A text format (for your data) is probably going to be just as good and will be human readable so you can look at the file and see what is wrong.
I am programming a face detection algorithm. In my code I'm parsing an XML file (in a recursion way, very inefficient takes my about 4 minutes to parse the whole XML file). I'd like to save the XML content using Iosteam binary to a file. I'm using a struct in C++ in order to use the raw data.
My goal is to parse the XML only if the raw data file is not exist.
The method work like this:
If the raw data file is not exist, parse the XML file and save the data to a file.
If the raw data file exist, read the raw data from the file
My problem is: whenever I open the raw data file and read from it. I get to read only small amount of byte from the file, I don't know how much, but in a certain point I receive only 0x00 data on my buffer.
My guess: I believe this has to do with the OS buffer, Which has a certain amount of buffer for read and write operations. I might be wrong about this. Though I'm not sure which one from the operations doesn't work well, it's either the write or read.
I was thinking to write / read the raw data char by char or line by line. In the other hand the file doesn't contain a text, which means that I can't read line by line or char by char.
The raw data size is
size_t datasize = DataSize(); == 196876 (Byte)
Which is retrieve in this function
/* Get the upper bound for predefined cascade size */
size_t CCacadeInterpreter::DataSize()
{
// this is an upper boundary for the whole hidden cascade size
size_t datasize = sizeof(HaarClassifierCascade) * TOTAL_CASCADE+
sizeof(HaarStageClassifier)*TOTAL_STAGES +
sizeof(HaarClassifier) * TOTAL_CLASSIFIERS +
sizeof(void*)*(TOTAL_CASCADE+TOTAL_STAGES+TOTAL_CLASSIFIERS);
return datasize;
}
The method work like this
BYTE * CCacadeInterpreter::Interpreter()
{
printf("|Phase - Load cascade from memory | CCacadeInterpreter::Interpreter | \n");
size_t datasize = DataSize();
// Create a memory structure
nextFreeSpace = pStartMemoryLocation = new BYTE [datasize];
memset(nextFreeSpace,0x00,datasize);
// Try to open a predefined cascade file on the current folder (instead of parsing the file again)
fstream stream;
stream.open(cascadeSavePath); // ...try existing file
if (stream.is_open())
{
stream.seekg(0,ios::beg);
stream.read((char*)pStartMemoryLocation , datasize); // **ream from file**
stream.close();
printf("|Load cascade from saved memory location | CCacadeInterpreter::Interpreter | \n");
printf("Completed\n\n");
stream.close();
return pStartMemoryLocation;
}
// Open the cascade file and parse the cascade xml file
std::fstream cascadeFile;
cascadeFile.open(cascadeDestanationPath, std::fstream::in); // open the file with read only attributes
if (!cascadeFile.is_open())
{
printf("Error: couldn't open cascade XML file\n");
delete pStartMemoryLocation;
return NULL;
}
// Read the file XML file , line by line
string buffer, str;
getline(cascadeFile,str);
while(cascadeFile)
{
buffer+=str;
getline(cascadeFile,str);
}
cascadeFile.close();
split(buffer, '<',m_tokens);
// Parsing begins
pHaarClassifierCascade = (HaarClassifierCascade*)nextFreeSpace;
nextFreeSpace += sizeof(HaarClassifierCascade);
pHaarClassifierCascade->count=0;
pHaarClassifierCascade->orig_window_size_height=20;
pHaarClassifierCascade->orig_window_size_width=20;
m_deptInTree=0;
m_numOfStage = 0;
m_numOfTotalClassifiers=0;
while (m_tokens.size())
{
Parsing();
}
// Save the current cascade into a file
SaveBlockToMemory(pStartMemoryLocation,datasize);
printf("\nCompleted\n\n");
return pStartMemoryLocation;
}
bool CCacadeInterpreter::SaveBlockToMemory(BYTE * pStartMemoryLocation,size_t dataSize)
{
fstream stream;
if (stream.is_open() )
stream.close();
stream.open(cascadeSavePath); // ...try existing file
if (!stream.is_open()) // ...else, create new file...
stream.open(cascadeSavePath, ios_base::in | ios_base::out | ios_base::trunc);
stream.seekg(0,ios::beg);
stream.write((char*)pStartMemoryLocation,dataSize);
stream.close();
return true;
}
Try using the Boost IOstreams library.
It has an easy to use wrrapers for file handling
I am working on an assignment for my GUI programming class, in which we are to make a windows program that displays the contents of a file in hexadecimal. I have a class that holds the text and creates the hex in string format.
I'm attempting to create an array of character arrays to store each line for output. However, when I use new to create the array of character pointers, I get an access violation error.
I've done some searching, but haven't had any luck finding the answer.
The class has these member variables:
char* fileText;
char** Lines;
int numChars;
int numLines;
bool fileCopied;
My constructor:
Text::Text(char* fileName){ //load and copy file.
fileText = NULL;
Lines = NULL;
fileCopied = ExtractText(fileName);
if ( fileCopied ) {
CreateHex();
}//endif
}//end constructor
ExtractText loads the file given to the constructor, and copies it into a large string.
bool Text::ExtractText(char fileName[]){
char buffer = '/0'; //buffer for text transfer
numChars = 0; //initialize numLines
ifstream fin( fileName, ios::in|ios::out ); //load file stream
if ( !fin ) { //return false if the file fails to load
return false;
}//endif
while ( !fin.eof() ) { //count the lines in the file
fin.get(buffer);
numChars++;
}//endwh
fileText = new char[numLines]; //create an array of strings, one for each line in the file.
fin.clear(); //clear the eof flag
fin.seekg(0, ios::beg); //move the get pointer back to the start of the file.
for ( int i = 0; i < numChars; i++ ) { //copy the text from the file into the string array.
fin.get(fileText[i]);
}//endfr
fileText[numChars-1] = '\0';
fin.close();
numLines = (numChars % 16 == 0) ? (numChars/16) : (numChars/16 + 1);
return true;
}//end fun ExtractText
Then comes the problem code. In the CreateHex function, the first line is where try to create the array of character pointers.
void Text::CreateHex(){
Lines = new char*[numLines];
As soon as the program runs that line of code, that's when I get the access violation. I'm not really sure what the problem is, because I've used that exact same method before in a previous program. The only difference was the name of pointer. I'm using Borland C++ 5.02 if that makes any difference. It's not my first choice in compilers, but its what our teacher wants us to use.
When you execute the line
fileText = new char[numLines]
The variable numLines has not yet been initialized. As a member variable, it's initialized to 0, so you are allocating an empty array for fileText.