Load file data into memory and saving it back out - c++

I'm working on a rough storage system for a small personal project. I have a struct that holds data for each stored file:
struct AssetTableRow {
std::string id = "Unnamed"; // a unique name given by the user
std::string guid = ""; // a guid generated based on the file data, used to detect duplicate files
std::string data; // binary data of the file
};
I load a file into it like this:
std::streampos size;
char* memblock;
std::ifstream file(filePath, std::ios::in | std::ios::binary | std::ios::ate);
if (file.is_open()) {
size = file.tellg();
memblock = new char[size];
file.seekg(0, std::ios::beg);
file.read(memblock, size);
file.close();
AssetTableRow row = AssetTableRow();
row.id = "myid";
row.guid = "myguid";
row.data = std::string(memblock);
AssetTable.push_back(row);
}
And then to try to write it back out to a file I used this:
std::ofstream file(destPath, std::ios::out | std::ios::binary);
if (file.is_open()) {
printf("Writing...\n", id.c_str());
// I think this is where it might be messing up
file.write((row.data.c_str(), row.data.c_str().size());
file.close();
printf("Done!\n", id.c_str());
}
Now when I try to open the file that got written out (.png sprite sheet) the photo viewer tells me it can't open a file of that type (but opening the original is fine).
If I open up the 2 files in Notepad++ (original on the left) I can see that they are indeed very different, there is almost no data in the output file!
I'm guessing this has something to do with the length on the write or read, but I've tried every different possible value I can think of for them and it doesn't seem to change anything.
If I print the data out to the console after it's read from the original file it appears as it does in the written file, so that leads me to believe the problem is with how I'm reading the file, but I fail to see any problems with that part of the code.
What is wrong with how I'm reading the file that it doesn't appear to be reading the whole file?
Also please forgive any awful mistakes I've made in my code, I'm still learning c++ and don't fully understand some parts of it so my code may not be the best.
EDIT:
As per Superman's advice about strings, I changed my my code to use char* for holding the data instead.
struct AssetTableRow {
std::string id = "Unnamed"; // a unique name given by the user
std::string guid = ""; // a guid generated based on the file data, used to detect duplicate files
char* data; // binary data of the file
};
And changed the read function so it's reading into the struct's data member:
std::ifstream file(actualPath, std::ios::in | std::ios::binary | std::ios::ate);
if (file.is_open()) {
AssetTableRow row = AssetTableRow();
size = file.tellg();
row.data = new char[size];
file.seekg(0, std::ios::beg);
file.read(row.data, size);
file.close();
row.id = "myid";
row.guid = "myguid";
printf("%s\n", row.data);
}
However I still see the same output from when I was using a string, so now I'm even more confused as to why this is happening.
EDIT2:
Upon further investigation I found that the size variable for reading the file reports the correct size in bytes. So now my guess is that for whatever reason it's not reading in the whole file

Related

Reads data correctly but writes incorrectly

I am trying to read specific binary data (2 Bytes) of a file and this mission works well, the problem when rewriting that (2 Bytes) again in the same place. Unfortunately, it changes the entire file data to zeros.
Look at the following two screenshots:
Data before writing:
Data after writing:
The code:
bool myClass::countChanger() {
std::ifstream sggFileObj_r(this->sggFilePath, std::ios::binary);
if (!sggFileObj_r.is_open()) {
std::cerr << strerror(errno) << std::endl;
return false;
}
// Buffer variable
unsigned short count;
// Move the file pointer to offset 4
sggFileObj_r.seekg(4);
// Reading data
sggFileObj_r.read((char*)&count, sizeof(unsigned short));
sggFileObj_r.close();
//// ---------------------- ////
std::ofstream sggFileObj_w(this->sggFilePath, std::ios::binary | std::ios::app);
// Increase the buffer variable by one
count += 1;
// Move the file pointer again to offset 4
sggFileObj_w.seekp(4);
// Rewriting data again to the file after modification
sggFileObj_w.write((char*)&count, sizeof(unsigned short));
sggFileObj_w.close();
return true;
}
Why did that happen and how to resolve?
UPDATE:
I have appended std::ios::app to file mode, and zeros problem solved but the specific data that I want to update is not updated.
Using
std::ofstream sggFileObj_w(this->sggFilePath, std::ios::binary)
will wipe out the data in the file since that is what ofstream does by default. You can use
std::ofstream sggFileObj_w(this->sggFilePath, std::ios::binary | std::ios::app);
to stop the data from being overridden but the issue with this is that file stream starts at the end of the file and pretends like the rest of the file doesn't exist, so you can seek back to the beggining and overwrite its contents.
What you can do instead is use a fstream like
std::fstream sggFileObj_w(this->sggFilePath, std::ios::binary | std::ios::out | std::ios::in);
To open the file in binary mode from the beginning without losing any contents. Then you can seek to where you want to write into the file.

saving file with ofstream

I'm new to C++, I have an image named "test.jpg", i convert it to base64 and decode it again like this:
std::ifstream inputFile;
inputFile.open("test.jpg",std::ios::binary);
std::filebuf* pbuf = inputFile.rdbuf();
inputFile.seekg (0, ios::end);
int length = inputFile.tellg();
// allocate memory to contain file data
char* buffer=new char[length];
// get file data
pbuf->sgetn (buffer,length);
inputFile.close();
CBase64 base64;
string encodedData = base64.base64_encode((unsigned char*)buffer,length);
delete[] buffer;
string decodedData = base64.base64_decode(encodedData);
ofstream outPutFile;
outPutFile.open("test2.jpg",ios::binary | ios::out);
outPutFile.write(decodedData.c_str(), decodedData.length());
outPutFile.close();
the "test2.jpg" has exact same size as "test.jpg"(the original file) but, i can't open it.
i couldn't find what is the problem.
i got it working. i just replaced:
outPutFile.open("test2.jpg",ios::binary | ios::out);
with
outPutFile.open("test2.jpg", ios_base::out | ios_base::binary);
std::string path = "file.txt";
std::string cfgString = "data";
std::ofstream output(path.c_str(), ios_base::out | std::ios::binary);
if (output.is_open()) {
output.write(cfgString.data(), cfgString.length());
}
output.close();
Apparently, there is no superficial problem with your file writing logic even though there are some irregularities. The biggest problem is in your main logic.
The program seems to be simple program of copying a file. What you are doing is reading a file, converting its data to base64 string and then again decoding the data to std::string. Now one small problem. Conversion of base64 string cannot be successfully done into a null terminated ANSI string for obvious reasons that any 0 in decoded binary data will terminate the string prematurely. Secondly you are opening a file in binary mode to write but trying to write std::string in the file. But that doesn't matter as you data has already been corrupted in your previous operation.
To solve this, you can simply use file copying example as this or make sure you write only binary data with care to your output file which means read in binary from input file and write to output file the same buffer. No base64 encoding decoding is required.
it looks like you forgot to write
inputFile.seekg (0, ios::beg);
after getting file length. it means you try to read from the end of the file instead of its beginning.

Problems while trying to Read Binary File C++

I'm writing a simple console application in Visual Studio C++. I want to read a binary file with .cer extension to a byte array.
ifstream inFile;
size_t size = 0;
char* oData = 0;
inFile.open(path, ios::in|ios::binary);
if (inFile.is_open())
{
size = inFile.tellg(); // get the length of the file
oData = new char[size+1]; // for the '\0'
inFile.read( oData, size );
oData[size] = '\0' ; // set '\0'
inFile.close();
buff.CryptoContext = (byte*)oData;
delete[] oData;
}
But when I launch it, I receive in all the oData characters the same char, every time another one, For example:
oData = "##################################################...".
Then I tried another way:
std::ifstream in(path, std::ios::in | std::ios::binary);
if (in)
{
std::string contents;
in.seekg(0, std::ios::end);
contents.resize(in.tellg());
in.seekg(0, std::ios::beg);
in.read(&contents[0], contents.size());
in.close();
}
Now the content has very strange values: a part of the values is correct, and a part is negative and strange values (maybe it is related to signed char and unsigned char?).
Does anyone have any idea?
Thanks ahead!
Looking at the first version:
What makes you think that tellg gets the size of the stream? It does not, it returns the current read position. You then go on to give a pointer to your data to buff.CryptoContents and promptly delete the data pointed to! This is very dangerous practice; you need to copy the data, use a smart pointer or otherwise ensure the data has the correct lifespan. It is likely the deletion is stomping your data with a marker to show it has been deleted if you're running in debug mode which is why you are getting the stream of identical characters.
I suspect your suggestion about signed and unsigned may be correct for the second but I can't say without seeing your file and data.
You are setting CryptoContext to point to your data by byte pointer, and after that you delete that data!
buff.CryptoContext = (byte*)oData;
delete[] oData;
After this lines CryptoContext is pointing to released and invalid data. Just keep oData array longer in memory and delete it after you are done with decoding or whatever you are doing with it.

C++ Read Entire Text File

i try to read the entire text file using vc++ with this code
ifstream file (filePath, ios::in|ios::binary|ios::ate);
if (file.is_open())
{
size = (long)file.tellg();
char *contents = new char [size];
file.seekg (0, ios::beg);
file.read (contents, size);
file.close();
isInCharString("eat",contents);
delete [] contents;
}
but it's not fetch all entire file ,why and how to handle this?
Note : file size is 1.87 MB and 39854 line
You are missing the following line
file.seekg (0, file.end);
before:
size = file.tellg();
file.seekg (0, file.beg);
As discribed in this example: http://www.cplusplus.com/reference/istream/istream/read/
Another way to do this is:
std::string s;
{
std::ifstream file ("example.bin", std::ios::binary);
if (file) {
std::ostringstream os;
os << file.rdbuf();
s = os.str();
}
else {
// error
}
}
Alternatively, you can use the C library functions fopen, fseek, ftell, fread, fclose. The c-api can be faster in some cases at the expense of a more STL interface.
You really should get the habit of reading documentation. ifstream::read is documented to sometimes not read all the bytes, and
The number of characters successfully read and stored by this function
can be accessed by calling member gcount.
So you might debug your issues by looking into file.gcount() and file.rdstate(). Also, for such big reads, using (in some explicit loop) the istream::readsome member function might be more relevant. (I would suggest reading by e.g. chunks of 64K bytes).
PS it might be some implementation or system specific issue.
Thanks all, i found the error where,
simply the code below reads the entire file ,
the problem was in VS watcher itself it was just display certain amount of data not the full text file.

Read/write operation works neither good nor bad

I am programming a face detection algorithm. In my code I'm parsing an XML file (in a recursion way, very inefficient takes my about 4 minutes to parse the whole XML file). I'd like to save the XML content using Iosteam binary to a file. I'm using a struct in C++ in order to use the raw data.
My goal is to parse the XML only if the raw data file is not exist.
The method work like this:
If the raw data file is not exist, parse the XML file and save the data to a file.
If the raw data file exist, read the raw data from the file
My problem is: whenever I open the raw data file and read from it. I get to read only small amount of byte from the file, I don't know how much, but in a certain point I receive only 0x00 data on my buffer.
My guess: I believe this has to do with the OS buffer, Which has a certain amount of buffer for read and write operations. I might be wrong about this. Though I'm not sure which one from the operations doesn't work well, it's either the write or read.
I was thinking to write / read the raw data char by char or line by line. In the other hand the file doesn't contain a text, which means that I can't read line by line or char by char.
The raw data size is
size_t datasize = DataSize(); == 196876 (Byte)
Which is retrieve in this function
/* Get the upper bound for predefined cascade size */
size_t CCacadeInterpreter::DataSize()
{
// this is an upper boundary for the whole hidden cascade size
size_t datasize = sizeof(HaarClassifierCascade) * TOTAL_CASCADE+
sizeof(HaarStageClassifier)*TOTAL_STAGES +
sizeof(HaarClassifier) * TOTAL_CLASSIFIERS +
sizeof(void*)*(TOTAL_CASCADE+TOTAL_STAGES+TOTAL_CLASSIFIERS);
return datasize;
}
The method work like this
BYTE * CCacadeInterpreter::Interpreter()
{
printf("|Phase - Load cascade from memory | CCacadeInterpreter::Interpreter | \n");
size_t datasize = DataSize();
// Create a memory structure
nextFreeSpace = pStartMemoryLocation = new BYTE [datasize];
memset(nextFreeSpace,0x00,datasize);
// Try to open a predefined cascade file on the current folder (instead of parsing the file again)
fstream stream;
stream.open(cascadeSavePath); // ...try existing file
if (stream.is_open())
{
stream.seekg(0,ios::beg);
stream.read((char*)pStartMemoryLocation , datasize); // **ream from file**
stream.close();
printf("|Load cascade from saved memory location | CCacadeInterpreter::Interpreter | \n");
printf("Completed\n\n");
stream.close();
return pStartMemoryLocation;
}
// Open the cascade file and parse the cascade xml file
std::fstream cascadeFile;
cascadeFile.open(cascadeDestanationPath, std::fstream::in); // open the file with read only attributes
if (!cascadeFile.is_open())
{
printf("Error: couldn't open cascade XML file\n");
delete pStartMemoryLocation;
return NULL;
}
// Read the file XML file , line by line
string buffer, str;
getline(cascadeFile,str);
while(cascadeFile)
{
buffer+=str;
getline(cascadeFile,str);
}
cascadeFile.close();
split(buffer, '<',m_tokens);
// Parsing begins
pHaarClassifierCascade = (HaarClassifierCascade*)nextFreeSpace;
nextFreeSpace += sizeof(HaarClassifierCascade);
pHaarClassifierCascade->count=0;
pHaarClassifierCascade->orig_window_size_height=20;
pHaarClassifierCascade->orig_window_size_width=20;
m_deptInTree=0;
m_numOfStage = 0;
m_numOfTotalClassifiers=0;
while (m_tokens.size())
{
Parsing();
}
// Save the current cascade into a file
SaveBlockToMemory(pStartMemoryLocation,datasize);
printf("\nCompleted\n\n");
return pStartMemoryLocation;
}
bool CCacadeInterpreter::SaveBlockToMemory(BYTE * pStartMemoryLocation,size_t dataSize)
{
fstream stream;
if (stream.is_open() )
stream.close();
stream.open(cascadeSavePath); // ...try existing file
if (!stream.is_open()) // ...else, create new file...
stream.open(cascadeSavePath, ios_base::in | ios_base::out | ios_base::trunc);
stream.seekg(0,ios::beg);
stream.write((char*)pStartMemoryLocation,dataSize);
stream.close();
return true;
}
Try using the Boost IOstreams library.
It has an easy to use wrrapers for file handling