How to update struct item in binary files - c++

Question

I have a binary file that i write some struct items to it. Now I want to find and update specific item from file items.
Note that my struct has a vector and its size is not constant.
my struct:
struct mapItem
{
string term;
vector<int> pl;
};
codes that write struct items to file
if (it==hashTable.end())//didn't find
{
vector <int> posting;
posting.push_back(position);
hashTable.insert ( pair<string,vector <int> >(md,posting ) );
mapItem* mi = new mapItem();
mi->term = md;
mi->pl = posting;
outfile.write((char*)mi, sizeof(mi));
}
else//finded
{
}
In else block I want to find and update item with its term(term is unique).
Now I have changed my code like this to serialize my vector.
if (it==hashTable.end())//didn't find
{
vector <int> posting;
posting.push_back(position);
hashTable.insert ( pair<string,vector <int> >(md,posting ) );
mapItem* mi = new mapItem();
mi->term = md;
mi->pl = posting;
if(!outfile.is_open())
outfile.open("sample.dat", ios::binary | ios::app);
size_t size = mi->term.size() + 1;
outfile.write((char*)&size, sizeof(size) );
outfile.write((char*)mi->term.c_str(), size);
size = (int)mi->pl.size() * sizeof(int);
outfile.write((char*)&size, sizeof(size) );
outfile.write((char*)&mi->pl[0], size );
outfile.close();
}
else//finded
{
(it->second).push_back(position);
mapItem* mi = new mapItem();
size_t size;
if(!infile.is_open())
{
infile.open("sample.dat", ios::binary | ios::in);
}
do{
infile.read((char*)&size, sizeof(size) ); // string size
mi->term.resize(size - 1); // make string the right size
infile.read((char*)mi->term.c_str(), size); // may need const_cast
infile.read((char*)&size, sizeof(size) ); // vector size
mi->pl.resize(size / sizeof(int));
infile.read((char*)&mi->pl[0], size );
}while(mi->term != md);
infile.close();
}
Well, my main question still remains: how can I update the data that I found?
Is there a better way to find them?

Answer 1

I evaluated the following solutions:
update in a new file, rename it to the old one in the end
update in the same file with a stream with two file positions, read & write, but I didn't rapidly find support for such a thing
update in the same file with two streams, read & write, but the risk of underlying overwrite is too big for me (even if protected outside against overlaps)
So I choose the first one, the most straightforward anyway.
#include <string>
#include <vector>
#include <fstream>
#include <cstdio>
#include <assert.h>
I added the following function to your struct:
size_t SizeWrittenToFile() const
{
return 2*sizeof(size_t)+term.length()+pl.size()*sizeof(int);
}
The read & write functions are basically same as your, except I choose not to write to string:c_str() pointer (although this ugly solution should work on every known compiles).
bool ReadNext(std::istream& in, mapItem& item)
{
size_t size;
in.read(reinterpret_cast<char*>(&size), sizeof(size_t));
if (!in)
return false;
std::istreambuf_iterator<char> itIn(in);
std::string& out = item.term;
out.reserve(size);
out.clear(); // this is necessary if the string is not empty
for (std::insert_iterator<std::string> itOut(out, out.begin());
in && (out.length() < size); itIn++, itOut++)
*itOut = *itIn;
assert(in);
if (!in)
return false;
in.read(reinterpret_cast<char*>(&size), sizeof(size_t));
if (!in)
return false;
std::vector<int>& out2 = item.pl;
out2.resize(size); // unfortunately reserve doesn't work here
in.read(reinterpret_cast<char*>(&out2[0]), size * sizeof(int));
assert(in);
return true;
}
// a "header" should be added to mark complete data (to write "atomically")
bool WriteNext(std::ostream& out, const mapItem& item)
{
size_t size = item.term.length();
out.write(reinterpret_cast<const char*>(&size), sizeof(size_t));
if (!out)
return false;
out.write(item.term.c_str(), size);
if (!out)
return false;
size = item.pl.size();
out.write(reinterpret_cast<const char*>(&size), sizeof(size_t));
if (!out)
return false;
out.write(reinterpret_cast<const char*>(&item.pl[0]), size * sizeof(int));
if (!out)
return false;
return true;
}
The update functions look like this:
bool UpdateItem(std::ifstream& in, std::ofstream& out, const mapItem& item)
{
mapItem it;
bool result;
for (result = ReadNext(in, it); result && (it.term != item.term);
result = ReadNext(in, it))
if (!WriteNext(out, it))
return false;
if (!result)
return false;
// write the new item content
assert(it.term == item.term);
if (!WriteNext(out, item))
return false;
for (result = ReadNext(in, it); result; result = ReadNext(in, it))
if (!WriteNext(out, it))
return false;
// failure or just the end of the file?
return in.eof();
}
bool UpdateItem(const char* filename, const mapItem& item)
{
std::ifstream in(filename);
assert(in);
std::string filename2(filename);
filename2 += ".tmp";
std::ofstream out(filename2.c_str());
assert(out);
bool result = UpdateItem(in, out, item);
// close them before delete
in.close();
out.close();
int err = 0;
if (result)
{
err = remove(filename);
assert(!err && "remov_140");
result = !err;
}
if (!result)
{
err = remove(filename2.c_str());
assert(!err && "remov_147");
}
else
{
err = rename(filename2.c_str(), filename);
assert(!err && "renam_151");
result = !err;
}
return result;
}
Questions ?

Answer 2

This:
outfile.write((char*)mi, sizeof(mi));
Does not make sense. You're writing the bits of a vector's implementation directly to disk. Some of those bits are extremely likely to be pointers. Pointers written to a file on disk are not useful, because they point to an address space belonging to the process which wrote the file, but won't work in another process reading the same file.
You need to "serialize" your data to the file, e.g. in a for loop writing each element.

Answer 3

You can serialize the struct to a file this way:
write length of string (4 bytes)
write string itself.
write length of vector (in bytes is easier to parse later).
write vector data. &vec[0] is the address of the first element. you can write all elements in ones shot since this buffer is contiguous.
Write:
size_t size = mi->term.size() + 1;
outfile.write((char*)&size, sizeof(size) );
outfile.write((char*)mi->term.c_str(), size);
size = (int)mi->pl.size() * sizeof(int);
outfile.write((char*)&size, sizeof(size) );
outfile.write((char*)&mi->pl[0], size );
Read:
infile.read((char*)&size, sizeof(size) ); // string size
mi->term.resize(size - 1); // make string the right size
infile.read((char*)mi->term.c_str(), size); // may need const_cast
infile.read((char*)&size, sizeof(size) ); // vector size
mi->pl.resize(size / sizeof(int));
infile.read((char*)&mi->pl[0], size );