To explain briefly: Using <fstream>, I write a std::list instance to a .txt file:
#include <fstream>
#include <list>
std::list<Item> list_1; //example list
list_1.push_back(Item(...));
std::ofstream file;
file.open("record.txt", std::ios::trunc);
if (file.is_open()) {
file.write((char*)&list_1, sizeof(std::list<Item>)) << std::endl;
file.close();
}
However, when I read from the same file and assign the data to a std::list instance:
file.open("record.txt", std::ios::in);
if (file.is_open()) {
std::list<Item> list_1;
file.read((char*)&list_1, sizeof(std::list<Item>));
}
It gives me an error when I try to access its elements. This is, however, not my problem. Because std::list stores the pointer to that element, I must store the elements manually, like I did here:
for (auto const& item : list_1) {
file << item.amount << std::endl;
file << item.value << std::endl;
file << item.item_name << std::endl;
file << (char*)&item.type << std::endl;
}
Then I read these values. Use the values to create a new Item instance and store it inside my list. Side note: I can access the size() property of the list_1 from the .txt file because it is a member of std::list<Item> which lives on the stack. So it gets saved by the ofstream.
for (int i = 0; i < list_1.size(); i++) {
int amount = 0;
int value = 0;
std::string item_name;
Item_Type type = item;
file >> amount;
file >> value;
file >> item_name;
file >> (char*)&type;
Item item(amount, value, item_name, type);
main_player_inv.push_back(item);
I expect this to work, because now the std::list should have no uninitialized members, right?
Well, it gives me this error:
this->_Mypair._Myval2._Myhead was 0x228B4050240
This basically means list_1->_Mypair._Myval2._Myhead is a pointer which points to memory out of bounds. The problem is, unlike the element pointers which I can manually save the values of and initialize, I can't access the data of list_1->_Mypair._Myval2._Myhead or edit it manually, as it is a private member. Or, there isn't a way I could find online.
So, I have two questions:
Can I initialize list_1->_Mypair._Myval2._Myhead so that it points to a valid memory?
Is there a way to more easily serialize a std::list and retrieve it's content?
If both of these questions are unanswerable, I would like to talk about what I'm trying to do:
The std::list<Item> is used as a character or an object's inventory. In my project, I want to store the items the player and objects such as containers have in a std::list<Item> instance. I thought this was the most fitting thing to do for an object-oriented Player structure. Here are some classes, for example:
Player class
class Player : public Object {
public:
int level, experience;
double health;
float resistance; // 0.000 - 1.000
std::list<Item> inventory;
public:
Player() :
level(0), experience(0), health(10.0), resistance(0.0f) {};
Player(int x, int y, std::string obj_name, Obj_Type type, int level, int experience, double health, float resistence) :
Object(x, y, obj_name, type), level(level), experience(experience), health(health), resistance(resistence) {};
};
Item class
struct Item {
public:
unsigned int amount, value;
std::string item_name;
Item_Type type; // enum
public:
Item() :
amount(0), value(0), item_name("undefined"), type(item) {};
Item(int amount, int value, std::string item_name, Item_Type type) :
amount(amount), value(value), item_name(item_name), type(type) {};
};
If you know a better way to store player items, items being class instances; or know altogether a better way to do this, please help me.
You can't read/write the raw bytes of a std::list object (or any other non-trivial type), as you would be writing/reading raw pointers and other internal data members that you don't need to concern yourself with.
You must (de)serialize your class's individual data members instead, as you have already discovered.
I would suggest a binary format instead of a textual format, eg:
#include <type_traits>
template <typename T, std::enable_if_t<std::is_scalar<T>::value, bool> = true>
void writeToStream(std::ostream &out, const T &value) {
out.write(reinterpret_cast<const char*>(&value), sizeof(value));
}
template <typename T, std::enable_if_t<std::is_scalar<T>::value, bool> = true>
void readFromStream(std::istream &in, T &value) {
in.read(reinterpret_cast<char*>(&value), sizeof(value));
}
void writeToStream(std::ostream &out, const std::string &value) {
size_t size = value.size();
writeToStream(out, size);
out.write(value.c_str(), size);
}
void readFromStream(std::istream &in, std::string &value) {
size_t size;
readFromStream(in, size);
value.resize(size);
in.read(value.data() /* or: &value[0] */, size);
}
template <typename Container>
void writeToStream(std::ostream &out, const Container &items) {
size_t count = items.size();
writeToStream(out, count);
for(const auto& item : items) {
writeToStream(out, item);
}
}
template <typename Container>
void readFromStream(std::istream &in, Container &items) {
size_t count;
readFromStream(in, count);
items.reserve(count);
for(size_t i = 0; i < count; ++i) {
Container::value_type item;
readFromStream(in, item);
items.push_back(item);
}
}
template<typename Container>
void writeToFile(const std::string &fileName, const Container &items) {
std::ofstream file(fileName, std::ios::binary);
file.exceptions(std::ofstream::failbit);
writeToStream(file, items);
}
template<typename Container>
void readFromFile(const std::string &fileName, Container &items) {
std::ifstream file(fileName, std::ios::binary);
file.exceptions(std::ifstream::failbit);
readFromStream(file, items);
}
struct Item {
public:
unsigned int amount, value;
std::string item_name;
Item_Type type; // enum
public:
Item() :
amount(0), value(0), item_name("undefined"), type(item) {};
Item(int amount, int value, std::string item_name, Item_Type type) :
amount(amount), value(value), item_name(item_name), type(type) {};
void writeTo(std::ostream &out) const {
writeToStream(out, amount);
writeToStream(out, value);
writeToStream(out, item_name);
writeToStream(out, type);
}
void readFrom(std::istream &in) {
readFromStream(in, amount);
readFromStream(in, value);
readFromStream(in, item_name);
readFromStream(in, type);
}
};
void writeToStream(std::ostream &out, const Item &item) {
item.writeTo(out);
}
void readFromStream(std::istream &in, Item &item) {
item.readFrom(in);
}
class Player : public Object {
public:
int level, experience;
double health;
float resistance; // 0.000 - 1.000
std::list<Item> inventory;
public:
Player() :
level(0), experience(0), health(10.0), resistance(0.0f) {};
Player(int x, int y, std::string obj_name, Obj_Type type, int level, int experience, double health, float resistence) :
Object(x, y, obj_name, type), level(level), experience(experience), health(health), resistance(resistence) {};
void writeTo(std::ostream &out) const {
writeToStream(out, level);
writeToStream(out, experience);
writeToStream(out, health);
writeToStream(out, resistance);
writeToStream(out, inventory);
}
void readFrom(std::istream &in) {
readFromStream(in, level);
readFromStream(in, experience);
readFromStream(in, health);
readFromStream(in, resistance);
readFromStream(in, inventory);
}
};
void writeToStream(std::ostream &out, const Player &player) {
player.writeTo(out);
}
void readFromStream(std::istream &in, Player &player) {
player.readFrom(in);
}
#include <fstream>
#include <list>
int main() {
std::list<Item> list_1; //example list
list_1.push_back(Item(...));
writeToFile("record.txt", list_1);
list_1.clear();
readFromFile("record.txt", list_1);
return 0;
}
If you really want a textual file, then use operator<< and operator>> instead, overriding them in your classes, eg:
(feel free to tweak this to use whatever formatting you want...)
#include <limits>
void discardLine(std::istream &in) {
in.ignore(std::numeeric_limits<std::streamsize>::max(), '\n');
}
template<typename CharT, typename Traits>
void streamFailed(std::basic_ios<CharT,Traits> &stream) {
stream.setstate(std::ios_base::failbit);
}
template <typename Container>
std::ostream& operator<<(std::ostream &out, const Container &items) {
out << '[' << items.size() << '\n';
for(const auto& item : items) {
out << item << '\n';
}
out << ']\n';
return out;
}
template <typename Container>
std::istream& operator>>(std::istream &in, Container &items) {
char ch;
in >> ch;
if (ch != '[') {
streamFailed(in);
} else {
size_t count;
in >> count;
discardLine(in);
items.reserve(count);
for(size_t i = 0; i < count; ++i) {
Container::value_type item;
in >> item;
items.push_back(item);
}
in >> ch;
if (ch != '[') {
streamFailed(in);
}
}
}
template<typename Container>
void writeToFile(const std::string &fileName, const Container &items) {
std::ofstream file(fileName, std::ios::binary);
file.exceptions(std::ofstream::failbit);
file << items;
}
template<typename Container>
void readFromFile(const std::string &fileName, Container &items) {
std::ifstream file(fileName, std::ios::binary);
file.exceptions(std::ifstream::failbit);
file >> items;
}
struct Item {
public:
unsigned int amount, value;
std::string item_name;
Item_Type type; // enum
public:
Item() :
amount(0), value(0), item_name("undefined"), type(item) {};
Item(int amount, int value, std::string item_name, Item_Type type) :
amount(amount), value(value), item_name(item_name), type(type) {};
friend std::ostream& operator<<(std::ostream &out, const Item &item) {
out << '(' << item.amount << ' ' << item.value << ' ' << static_cast<int>(item.type) << ' ' << item.item_name << ')';
return out;
}
friend std::istream& operator>>(std::istream &in, Item &item) {
char ch;
in >> ch;
if (ch != '(') {
streamFailed(in);
} else {
int itype;
in >> item.amount >> item.value >> itype;
item.type = static_cast<Item_Type>(itype);
std::getline(in >> std::ws, item_name, ')');
}
return in;
}
};
class Player : public Object {
public:
int level, experience;
double health;
float resistance; // 0.000 - 1.000
std::list<Item> inventory;
public:
Player() :
level(0), experience(0), health(10.0), resistance(0.0f) {};
Player(int x, int y, std::string obj_name, Obj_Type type, int level, int experience, double health, float resistence) :
Object(x, y, obj_name, type), level(level), experience(experience), health(health), resistance(resistence) {};
friend std::ostream& operator<<(std::ostream &out, const Player &player) {
out << '(' << level << ' ' << experience << ' ' health << ' ' << resistance << '\n';
out << inventory;
out << ')';
return out;
}
friend std::istream& operator>>(std::istream &in, Player &player) {
char ch;
in >> ch;
if (ch != '(') {
streamFailed(in);
} else {
in >> player.level >> player.experience >> player.health >> player.resistance >> player.inventory;
in >> ch;
if (ch != ')') {
streamFailed(in);
}
}
return in;
}
};
#include <fstream>
#include <list>
int main() {
std::list<Item> list_1; //example list
list_1.push_back(Item(...));
writeToFile("record.txt", list_1);
list_1.clear();
readFromFile("record.txt", list_1);
return 0;
}
i am dealing with large dataset. May i ask you how it is possible to store strings in the classes i want to use with stxxl? I have read several discussions and everywhere was said that string is not POD type therefore it cannot be stored in the stxxl::vector, but i am not sure,because i tried it and i checked the data and everything seems to be fine. i have also saw an approach here https://github.com/Project-OSRM/osrm-backend/blob/725b93a961625a7b04d54806d7e0f80252a6bcd0/extractor/extraction_containers.hpp and they use stxxl::vector, so maybe the library got updated to support std::string?
class HighWay
{
private:
uint64_t id;
string name;
int speed;
string attributes; //additional attributes of way
string edges; //written uint64_t from,uint64_t to, int distance written as string
string nodes; //vector<uint64_t> written as string
public:
HighWay() = default;
void setId(uint64_t _id) {
id = boost::lexical_cast<string>(_id);
}
void setName(string _name) {
name = _name;
}
void setSpeed(int _speed) {
speed = _speed;
}
void setAttributes(string _attributes) {
attributes = _attributes;
}
void setEdges(string _edges) {
edges = _edges;
}
void setNodes(vector<uint64_t>refs) {
stringstream s;
uint64_t i = 0;
for (; i < refs.size()-1;i++) {
s << boost::lexical_cast<uint64_t>(refs[i]) << " ";
}
s << boost::lexical_cast<uint64_t>(refs[i]);
nodes = s.str();
}
uint64_t getId() {
return id;
}
string getName() {
return name;
}
int getSpeed() {
return speed;
}
string getAttributes() {
return attributes;
}
string getEdges() {
return edges;
}
std::vector<int64_t> getNodes() {
stringstream s(nodes);
uint64_t node;
std::vector<int64_t> result;
while (s >> node) {
result.push_back(static_cast<int64_t>(node));
}
return result;
}
};
I have also created code which stores the strings as POD,storing the string in vector of char and in map remembering lower and upper bound index in the array. But this approach leads to many std::maps used in the application.
//class to store in map
struct TypeName{
uint64_t start;
uint64_t end;
};
std::istream& operator >> (std::istream& i, TypeName& entry)
{
i >> entry.start;
i >> entry.end;
return i;
}
std::ostream& operator << (std::ostream& i, const TypeName& entry)
{
i << entry.start << " ";
i << entry.end;
return i;
}
struct PoiCategories{
uint64_t start;
uint64_t end;
};
std::istream& operator >> (std::istream& i,PoiCategories& entry)
{
i >> entry.start;
i >> entry.end;
return i;
}
std::ostream& operator << (std::ostream& i, const PoiCategories& entry)
{
i << entry.start << " ";
i << entry.end;
return i;
}
//object i want to store
struct Poi {
Poi() = default;
uint64_t id;
char type;
uint64_t id_in_pois; //id in vector pois
void addCategories(
vector<int> &kats, //categories to insert
stxxl::vector<uint64_t> &categories, //vector to store category
std::unordered_map <uint64_t, PoiCategories> &idPoi_categories //index to vector categories to retrieve all categories for Poi
)
{
size_t start = categories.size();
for (auto & kat : kats) {
categories.push_back(kat);
}
size_t end = categories.size() - 1;
idPoi_categories.insert(make_pair(id, PoiCategories{start, end }));
}
vector<int> getCategories(
stxxl::vector<uint64_t> &categories,
std::unordered_map <uint64_t, PoiKategorie> &idPoi_categories
)
{
std::vector<int> result;
PoiCategories bounds = idPoi_categories.find(id)->second;
for (size_t i = bounds.start; i <= bounds.end; i++) {
result.push_back(categories[i]);
}
return result;
}
};
Problem in my application is that i am storing a few string data, which are mainly names of streets and POIs. Maybe i am using wrong library. If so,can you recommend me a better approach to store data while preprocessing?
It's indeed banned, but the symptoms of violating the no-POD rule are generally unpredictable. It may very well appear to work as long as the strings all fit in memory, but in that case you didn't need the STXXL anyway.
Given the following scenario where my data might be of different type based on some condition.
class myClass {
public:
myclass() {
if (condition1) {
bool boolValue = false;
data = boolValue;
} else if (condition2) {
int intValue = 0;
data = intValue;
} else if (condition3) {
unsigned int unsignedIntValue = 0;
data = unsignedIntValue;
} else if (condition4) {
long longValue = 0;
data = longValue;
} else if (condition5) {
double doubleValue = 0.0;
data = doubleValue;
} else if (condition6) {
float floatValue = 0.0;
data = floatValue;
} else if (condition7) {
char *buffer = new char[10];
data = buffer;
}
}
void* getData() const { return data; }
private:
void *data;
}
As it happens the value that my void pointer points to is strictly within each statement. Therefore what is returned with getData() might not be valid. If I do get the data it is simply because the memory location where I point to is not yet written over.
The solution I have come up with is this:
class myClass {
public:
myclass() {
if (condition1) {
boolValue = false;
data = boolValue;
} else if (condition2) {
intValue = 0;
data = intValue;
} else if (condition3) {
unsignedIntValue = 0;
data = unsignedIntValue;
} else if (condition4) {
longValue = 0;
data = longValue;
} else if (condition5) {
doubleValue = 0.0;
data = doubleValue;
} else if (condition6) {
floatValue = 0.0;
data = floatValue;
} else if (condition7) {
buffer = new char[10];
data = buffer;
}
}
void* getData() const { return data; }
private:
void *data;
bool boolValue;
int intValue;
unsigned int unsignedIntValue;
long longValue;
double doubleValue;
float floatValue;
char *buffer;
}
I was thinking there must be a more elegant way to do this. Any suggestions?
You could use a union to save a few bits in memory, and then use pointer casting to get the value from the union:
#include<iostream>
using namespace std;
class myClass {
public:
myClass(char *str){
data.str = str;
}
myClass(double d){
data.d = d;
}
myClass(float f){
data.f = f;
}
void *getData() { return (void*)&data; }
private:
union {
double d;
float f;
char *str;
} data;
};
int main(){
myClass c(2.0);
cout << *(double*)c.getData() << endl;
myClass f(3.0f);
cout << *(float*)f.getData() << endl;
myClass s("test");
cout << *(char**)s.getData() << endl;
system("pause");
}
/* prints
2
3
test
*/
If you don't need to change the type of the data after you create an object, then you could use a template class:
template <typename T>
class myBaseClass {
public:
// Declare common functions here.
T getData()
{ return data; }
protected:
T data;
protected:
// Disallow constructing instances of this class outside the child classes.
myBaseClass(T val) : data(val) { }
};
template <typename T>
class myClass: public myBaseClass<T> {
public:
myClass() : myBaseClass<T>(0) { }
};
You then specialize for char*:
template <>
class myClass<char*>: public myBaseClass<char*> {
public:
myClass() : myBaseClass(new char[10]) { }
};
You then create instances like this:
myClass<int> a;
myClass<float> b;
myClass<char*> c;
// etc.
int i = a.getData();
float f = b.getData();
char* str = c.getData();
I need start off with code because I am not sure what terminology to use. Lets say I have the following code:
class Node
{
public:
void Parse(rapidxml::xml_node<> *node)
{
for (rapidxml::xml_attribute<> *attr = node->first_attribute();
attr;
attr = attr->next_attribute())
{
std::stringstream converter;
converter << attr->value();
if( !strcmp(attr->name(), "x") ) converter >> x;
else if( !strcmp(attr->name(),"y") ) converter >> y;
else if( !strcmp(attr->name(), "z") ) converter >> z;
}
}
private:
float x;
float y;
float z;
};
What I can't stand is the repetition of if( !strcmp(attr->name(), "x") ) converter >> x; I feel that this is error prone and monotonous, but I cannot think of another way to map a string value to a member assignment. What are some other approaches one can take to avoid code such as this? The only other possible alternative I could think of was to use a hashmap, but that runs into problems with callbacks
This is the best I could up with but it's not as flexible as I'd like:
class Node
{
Node() : x(0.0f), y(0.0f), z(0.0f)
{
assignmentMap["x"] = &x;
assignmentMap["y"] = &y;
assignmentMap["z"] = &z;
}
public:
void Parse(rapidxml::xml_node<> *node)
{
for (rapidxml::xml_attribute<> *attr = node->first_attribute();
attr;
attr = attr->next_attribute())
{
map<std::string, float*>::iterator member = assignmentMap.find(attr->name());
//check for a pre-existing entry
if( member == assignmentMap.end()) continue;
std::stringstream converter;
converter << attr->value();
converter >> *(member->second);
}
}
private:
float x;
float y;
float z;
std::map<std::string, float*> assignmentMap;
};
For the implementation with a map, you could use pointers-to-members. Then you won't need a deep copy of the map (when you copy it, the pointers in the map still point into the original Node), and it will also allow you to make the whole thing static (this map is unnecessary at per-instance basis).
For example:
class Node {
//...
static std::map<std::string, float Node::*> initVarMap();
static float Node::* varFromName(const std::string& name);
};
std::map<std::string, float Node::*> Node::initVarMap()
{
std::map<std::string, float Node::*> varMap;
varMap["x"] = &Node::x;
varMap["y"] = &Node::y;
varMap["z"] = &Node::z;
return varMap;
}
float Node::* Node::varFromName(const std::string& name)
{
static std::map<std::string, float Node::*> varMap = initVarMap();
std::map<std::string, float Node::*>::const_iterator it = varMap.find(name);
return it != varMap.end() ? it->second : NULL;
}
Usage:
float Node::* member(varFromName(s));
if (member)
this->*member = xyz;
This isn't any more flexible, though.
To support different types of members, you might modify the above to use a map of string to "variant of all supported member types".
For example so. The member setter visitor should be reusable, and the only change to the code, to add or change member types, should be done to the typedef.
#include <map>
#include <string>
#include <iostream>
#include <boost/variant.hpp>
template <class Obj, class T>
struct MemberSetter: boost::static_visitor<void>
{
Obj* obj;
const T* value;
public:
MemberSetter(Obj* obj, const T* value): obj(obj), value(value) {}
void operator()(T Obj::*member) const
{
obj->*member = *value;
}
template <class U>
void operator()(U Obj::*) const
{
//type mismatch: handle error (or attempt conversion?)
}
};
class Node
{
public:
Node() : i(0), f(0.0f), d(0.0f)
{
}
template <class T>
void set(const std::string& s, T value)
{
std::map<std::string, MemberTypes>::const_iterator it = varMap.find(s);
if (it != varMap.end()) {
boost::apply_visitor(MemberSetter<Node, T>(this, &value), it->second);
} //else handle error
}
void report() const
{
std::cout << i << ' ' << f << ' ' << d << '\n';
}
private:
int i;
float f;
double d;
typedef boost::variant<int Node::*, float Node::*, double Node::*> MemberTypes;
static std::map<std::string, MemberTypes> initVarMap();
static std::map<std::string, MemberTypes> varMap;
};
int main()
{
Node a;
a.set("i", 3);
a.set("d", 4.5);
a.set("f", 1.5f);
a.report();
}
std::map<std::string, Node::MemberTypes> Node::initVarMap()
{
std::map<std::string, Node::MemberTypes> varMap;
varMap["i"] = &Node::i;
varMap["f"] = &Node::f;
varMap["d"] = &Node::d;
return varMap;
}
std::map<std::string, Node::MemberTypes> Node::varMap = Node::initVarMap();
This is naturally just an example of what you can do. You can write a static_visitor to do what you want. E.g storing a stream and attempting to extract a value of the right type for the given member.
Use an array. An alternative to this union would be to let x, y, and z be references (float&) to array elements 0, 1, 2 — or (my preference) always call them by number not by name.
class Node
{
public:
void Parse(rapidxml::xml_node<> *node)
{
std::stringstream converter;
for (rapidxml::xml_attribute<> *attr = node->first_attribute();
attr;
attr = attr->next_attribute())
{
if ( strlen( attr->name() ) != 1
|| *attr->name() < 'x' || *attr->name() > 'z' )
throw rapidxml::parse_error; // or whatever
converter << attr->value() >> ary[ *attr->name() - 'x' ];
}
}
private:
union {
float ary[3]; // this can come in handy elsewhere
struct {
float x;
float y;
float z;
} dim;
};