I am using the https://github.com/nlohmann/json libary for writing out my object to BSON. I have validated creating the json instance by writing the file out to json for review, so that portion of the code is working as expected. I am having issues with it once I swap back to bson though.
Here is the snippet of code I am working with:
//I have written out json_triangle_container to verify it looks correct
//so I am sure the problem is not here
json json_instance = <my function to create this instance>
std::vector<std::uint8_t> v_bson = json::to_bson(json_instance);
//writing the vector out here
auto bsonFilename = std::string("json_instance.bson");
std::ofstream bsonfile(bsonFilename, std::ios::out | std::ios::binary);
bsonfile.write((char*)&v_bson[0], v_bson.size() * sizeof(v_bson[0]));
bsonfile.close();
//reading it back in here
std::ifstream r_file(bsonFilename, std::ios::in | std::ios::binary | std::ios::ate);
std::streampos size = r_file.tellg();
r_file.seekg(0, std::ios::beg);
char* memblock = new char[size];
r_file.read(memblock, size);
r_file.close();
//printing out the json object for testing
auto json_instance_read = json::from_bson(memblock); //this is where it is blowing up
std::cout << json_instance_read << std::endl;
I get an exception that says:
Exception thrown at 0x00007FFF4EE6A839 in test.exe: Microsoft C++ exception: nlohmann::detail::parse_error at memory location 0x0000006AB4D5F470.
It is also worth noting if I call json::from_bson(v_bson ) it parses fine. I validated the array size being read in is the same size as the vector I wrote out. So this is leading me to believe I did something wrong in how I wrote the bson out initially.
Edit
Using some online tools to convert the bson file I generated to json it seems the writing is correct. So that points to my read function being the issue.
Doing some more debugging, it looks like in the v_bson vector the 3 item has the value '\0' which I think is throwing off the reader and it is stopping short before reading the entire file. I confirmed this by checking the size of the memblock and it is only 3 characters long.
So the question changes a bit:
How do I tell ifstream reader to ignore what it seem to think is an EOF character?
Everything I have found says to make sure to initialize it with std::ios::binary, which I do:
std::ifstream r_file(bsonFilename, std::ios::in | std::ios::binary | std::ios::ate);
Related
After calling seekp() to the middle of file, then writing my buffer, the file-pointer teleports to the end of file.
My code is more complex but essentially comes down to the following:
std::lock_guard<std::mutex> lckFile(_mu_fileAccess);
const char* buff = new const char[200];
int preallocSize = 1024*512;
_f.open(path_file_with_exten, std::ios::app | std::ios::binary );
if(!_f){
delete[] buff;
return;
}
std::experimental::filesystem::resize_file(path_with_exten, preallocSize);
_f.seekp(321, std::ios_base::beg);
int64_t currPos = _f.tellp(); //will be 321
_f.write(buff, 100);
_f.tellp(); //somehow is equal to 1024*512 instead of 321+100
What might be causing this behavior? It occurs only in one specific place of my code. Other invocations of this code from other places work ok and don't teleport the pointer that far away.
I'm using C++14
Edit:
Following the below answer, I had to use std::ios::ate not std::ios::app.
However, I discovered that my file started to be re-created from scratch even though I'm not specifying std::ios::trunc.
I had to pass in additional flag in to make sure the file contents are preserved:
auto flags = std::ios::ate | std::ios::in | std::ios::out | std::ios::binary; _f.open("myFile.exten", flags);
otherwise documentation says "Creating a file. If the file already exists, the old version
is deleted."
From my comment above.
Documentation for open modes
app: seek to the end of stream before each write
Thus :
_f.write(buff, 100);
…triggers a seek to the end of the file before writing.
I'm trying to load a .wav file but it takes forever. The code is use:
std::ifstream file (filePath, std::ios::binary);
// check if the file exists
if (! file.good())
{
reportError ("ERROR: File doesn't exist or otherwise can't load file\n" + filePath);
return false;
}
file.unsetf (std::ios::skipws);
std::istream_iterator<uint8_t> begin (file), end;
std::vector<uint8_t> fileData (begin, end);
As I said, it takes forever to load a 27MB file. But I have seen a few posts online that claim to have loaded a 106MB file in around 1400ms by using getline().
I know that ifstream::read() can load data faster, but then I wouldn't get the data as uint8_t back.
I haven't really worked much with files so it would be nice if somebody could explain how i could read the data fast and get it as uint8_t. Any help regarding this would be appreciated.
EDIT:
I now reserve space for the bytes, but it is still slow:
std::ifstream file(filePath, std::ios::binary);
auto size = std::filesystem::file_size(filePath);
// check if the file exists
if (!file.good())
{
reportError("ERROR: File doesn't exist or otherwise can't load file\n" + filePath);
return false;
}
file.unsetf(std::ios::skipws);
std::istream_iterator<uint8_t> begin(file), end;
std::vector<uint8_t> fileData;
fileData.reserve(size);
fileData = std::vector<uint8_t>(begin, end);
Any other ideas on how I might speed this up?
EDIT 2:
I searched online and found this:
char* cp = new char[100000000];
std::ofstream ofs("bff.txt"); //make a huge file to test with.
ofs.write(cp, 100000000);
ofs.close();
std::ifstream ifs("bff.txt");
ifs.read(cp, 100000000);
ifs.close();
This is extremely fast and also copies all the data into into the char array extremely quickly. Can anyone tell me how I might do this with uint8_ts?
The way to do this is indeed by using read, but to get your uint8_ts without having to cast any pointers, you can use std::basic_ifstream, which is a template. Using this lets you specify the pointer type.
So, quick proof-of-concept program (no error checking or anything, just to show you how it works):
#include <cstdint>
#include <fstream>
#include <vector>
int main ()
{
std::basic_ifstream <uint8_t> f ("myfile", std::ios::binary);
std::vector <uint8_t> v (100);
f.read (v.data (), v.size ());
}
And that's it!
Demo
I want to use seekp() function with binary files to modify the data of certain integer. I have made simple test program to test seekp funstion but it does not work rather it deletes the old content in file.
// position in output stream
#include <fstream> // std::ofstream
#include <iostream>
int main() {
std::ofstream outfile{"test.dat" ,std::ios::binary| std::ios::out};
struct t {
int x;
};
t t1{ 36};
t t2{ 2 };
t t3{ 3 };
outfile.seekp(4 , std::ios::beg );
outfile.write((char*)&t1 , sizeof(t1));
outfile.write((char*)&t2, sizeof(t2));
outfile.write((char*)&t3, sizeof(t3));
outfile.close();
std::ifstream iutfile{ "test.dat" ,std::ios::binary };
t read;
while (iutfile.read((char*)&read, sizeof(read)))
{
std::cout << "\n\n\t" << read.x;
}
iutfile.close();
return 0;
}
#here are my steps i am doing to test it:#
1)comment the outfile.seekp(4 , std::ios::beg ); in code above then it will print the contents in file
2)now uncomment the seekp line and comment the two outfile.write() lines leaving one to test whetehr seekp is putting the pointer so i can write at exact loaction But when i do this the previous data lost
3)i then tried commenting all write lines leaving seekp lines uncommented and then i saw the whole file cintent is removed
WHat i am doing wrong i cant understand . I have tried seekp(sizeof(int) , std::ios::beg) but it also doest not work .. Any help please
You destroy content of file every time you open it, it has nothing to do with seekp.
std::ios::out means implicitly std::ios::trunc, unless there is std::ios::in or std::ios::app as well (yes, it's a mess). See cppreference for detailed description of how flags work with each other.
You need to open file in append mode and then seekp to position that interests you:
std::ofstream outfile{"test.dat" ,std::ios::binary | std::ios::app}; //out is optional
std::ofstream outfile{"test.dat" ,std::ios::binary | std::ios::app | std::ios::out};
I try to replace data in binary file using C++ and fstream library. Is it possible to do that with this library?
I would like to change one byte of file in address: 0xB07 to 1.
I have written the following code snippet:
...
int address = 0xB07;
char * toChange = new char('0');
std::ifstream input(filename, std::ios::binary);
input.seekg(address);
input.read(toChange, 1);
input.close();
*toChange = (char)0x01;
std::ofstream output(filename, std::ios::binary);
output.seekp(address);
output.write(toChange, 1);
output.close();
...
I have tried many versions of this code and I still can't understand why the byte doesn't change.
This code will remove your file and put totally new contents in it.
The problem is in line
std::ofstream output(filename, std::ios::binary);
That's because the default open mode is ios::out | ios::trunc (See for ex.: Input/Output with files)
Which means file is being truncated to zero bytes.
Then your seek() function extends him to the specified size (usually filling with zeroes) and finally output.write() writes the byte at the end.
In order to do what you want I had to specify the following ofstream flags: std::ios::binary|std::ios::out|std::ios::in
I cannot say currently for sure why the std::ios::inis needed, sorry...
I am trying to create a method that copies a file to a folder that is local to my project. I am quite puzzled because from what I understand this should work. I decided to create a simple text file to test my copy file method but it doesn't seem to be working.
std::string newFile="Files\\newText.txt";
std::ifstream oldFile("C:\\Users\\dtruman\\Documents\\oldText.txt", std::ios::binary | std::ios::in);
std::ofstream newTarget(newFile, std::ios::binary | std::ios::out);
char c;
while(oldFile.get(c));
{
std::cout << c << std::endl;
newTarget.put(c);
}
newTarget.close();
oldFile.close();
Some of this stuff was me fiddling with the code. My problem is that no matter what I seem to do it never seems to copy the file over correctly, the contents of the new text file are always different then the original. Am I missing something, to my knowledge this block of code should work.
This line
while(oldFile.get(c));
consumes the entire file without any side effects due to the ; at the end.
You need:
while(oldFile.get(c)) // Without the ;
{
std::cout << c << std::endl;
newTarget.put(c);
}