Trigger for std::map operation - c++

I want to implement something like maplistner in c++ for std::map. So when a key is added or updated in std::map it should trigger a function or object.
It should be as:
class MapListener : public std::map
{
// ----- MapListener methods---
public:
// Invoked when a map entry has been inserted.
virtual void entryInserted();
// Invoked when a map entry has been updated.
virtual void entryUpdated();
// Invoked when a map entry has been removed.
virtual void entryDeleted();
};
Any help will be appreciated.
Thanks

Solution
(I am not advocating you should do this, just showing how to do so, because this sounds very much like the x-y problem to me).
template<class Key, class T, class Compare = std::less<Key>>
class EventMap final
{
public:
EventMap(): data_{}
{}
// std::map::insert looks like this:
// std::pair<iterator,bool> insert( const value_type& value );
using insert_callback = void (const value_type& inserted);
void on_insert(insert_callback* cb)
{
on_insert_ = cb;
}
// wrapped insert:
void insert(const value_type& value)
{
data_.insert(value);
if(on_insert_)
(*on_insert_)(value);
}
// TODO: add similar code for other methods w/ callbacks
private:
std::map<Key, T, Compare> data_;
insert_callback *on_insert_ = nullptr;
};
client callback:
auto on_insertion(const auto& kv)
{
std::cout << "inserted: " << kv.first << ", " << kv.second << "\n";
}
int main()
{
EventMap<std::string, std::string> map;
map.on_insert(&on_insertion);
map.insert({"123", 123}); // will call on_insertion after inserting
}

I apologise for blatantly ripping off #utnapistim solution, but the original would not compile. Nevertheless I find it interesting and would like to share working version:
#include <iostream>
#include <string>
#include <map>
template<class Key, class T, class Compare = std::less<Key>>
class EventMap final
{
public:
EventMap() : data_{}
{}
// std::map::insert looks like this:
// std::pair<iterator,bool> insert( const value_type& value );
using insert_callback = void(const std::pair<const Key, T> &inserted);
using erase_callback = void(const Key &key);
void on_insert(insert_callback* icb)
{
on_insert_ = icb;
}
void on_erase(erase_callback *ecb)
{
on_erase_ = ecb;
}
// wrapped insert:
void insert(const std::pair<const Key, T>&value)
{
data_.insert(value);
if (on_insert_)
(*on_insert_)(value);
}
// wrapped erase:
void erase(const Key &key)
{
data_.erase(key);
if (on_erase_)
(*on_erase_)(key);
}
// TODO: add similar code for other methods w/ callbacks
private:
std::map<const Key, T, Compare> data_;
insert_callback *on_insert_ = nullptr;
erase_callback *on_erase_ = nullptr;
};
int main()
{
EventMap<const std::string, std::string> map;
auto on_insert_fn = [](std::pair<const std::string, std::string> const &kv)
{
std::cout << "inserted: " << kv.first << ", " << kv.second << "\n";
};
auto on_erase_fn = [](const std::string &key)
{
std::cout << "erased: " << key << "\n";
};
map.on_insert(on_insert_fn);
map.insert({ "123", "456" }); // will call on_insert_fn after inserting
map.insert({ "786", "101112" }); // will call on_insert_fn after inserting
map.on_erase(on_erase_fn);
map.erase("123"); // will call on_erase_fn after erasing
system("pause");
return 0;
}

You absolutely don't want to have : public std::map in your class definition, because then there is no hope of keeping all of the ways of modifying the map contained.
What you will instead find is that you have one type for the client, containing the event handlers, and another type that wraps a map and delegates every action to it, plus calls the relevant handler function.
You won't be able to use this in lots of places where map would work, as to see modifications to values via iterators and operator[] you are forced to use a proxy for your reference member. Thus your iterator and const_iterator members don't satisfy ForwardIterator (and BidirectionalIterator). Thus you don't satisfy Container (and AssociativeContainer).
As an alternative to not being a Container, you could instead have a value_type of std::pair<const Key, const Value>, and simply require that modifications happen with an "erase-modify-insert" sequence.

Related

Visit all std::variants in a list and capture an additional value for every visit

I'm trying to implement a state store, which is basically a map of key-value pairs where the value is a std::variant.
The following class works fine. Note that it not only prints the visited value, but also the keys:
class StateStoreTest
{
typedef std::variant<float, std::string> StateValue;
private:
std::map<int, StateValue> States;
public:
void SetValue(int key, const StateValue& value) { States[key] = value; }
void VisitAllValues()
{
for (auto& it : States)
{
auto key = it.first;
std::visit([key](auto&& value) { std::cout << key << "=" << value << "\n"; }, it.second);
}
}
};
I can use it like this and the output is as expected:
StateStoreTest sst;
sst.SetValue(1, 123.4f);
sst.SetValue(2, "Test");
sst.VisitAllValues();
Now I don't want to implement a specific visitor implementation in VisitAllValues() but I want to be able to provide a visitor callable to the VisitAllValues() method. In that visitor callable I want to have access to both the key and also the visited value.
So basically from a caller perspective I'd like to be able to do something like this:
sst.VisitAllValues([](auto key, auto&& value){ std::cout << key << "=" << value << "\n"; });
I guess my VisitAllValues() method would have to look something like this:
void VisitAllValues(??? visitor)
{
for (auto& it : States)
{
auto key = it.first;
std::visit([key](auto&& value) { visitor(key, value); }, it.second);
}
}
But I couldn't figure out the details (I tried writing VisitAllValues() as a template, but then I ran into problems on the caller side).
How can I implement my VisitAllValues() method so it accepts a visitor which has access to both the id and the value?
Here is the full working toy example to play with.
It seems to me that as follows should works
template <typename L>
void VisitAllValues (L const & visitor)
{
for (auto& it : States)
{
auto key = it.first;
std::visit([&](auto&& value) { visitor(key, value); }, it.second);
}
}
Off Topic: maybe using std::forward is you use forwarding references
visitor(key, std::forward<decltype(value)>(value));

How emulate a templatised std::function in C++

Following is a basic instance of what I am doing in my C++ program. I have a list of listeners which are all std::functions. I have a concept DataType which means what kind of data the listener is interested in. The idea here is the same as publish-subscribe pattern. A method interested in certain kind of data should be able to add itself to the list of listeners using AddListener. Some methods are added & they receive a callback whenever required.
The program works fine !!
#include <iostream>
#include <functional>
#include <vector>
#include <string>
enum class DataType {
Type_1,
Type_2
// and so on
};
typedef std::function<void(std::pair<DataType, std::string>)> MyListenerType;
//template <typename T>
//typedef std::function<void(T>)> MyListenerType;
// How can I emulate the above so that a method passing any kind of primitive data-type namely "int, bool, float or double" can be added into
// my vector of listners.
std::vector<MyListenerType> my_data_listeners_1;
std::vector<MyListenerType> my_data_listeners_2;
void ListenerMethod_Instance_1(std::pair<DataType, std::string> information) {
DataType data_type = information.first;
std::string message = information.second;
std::cout << "ListenerMethod_Instance_1 called with message " << message << "\n";
}
void ListenerMethod_Instance_2(std::pair<DataType, std::string> information) {
DataType data_type = information.first;
std::string message = information.second;
std::cout << "ListenerMethod_Instance_2 called with message " << message << "\n";
}
void AddListener (MyListenerType listener, DataType type_of_interest) {
if (DataType::Type_1 == type_of_interest) {
my_data_listeners_1.push_back(listener);
std::cout << "Added a method instance for DataType::Type_1" << "\n";
}
else if (DataType::Type_2 == type_of_interest) {
my_data_listeners_2.push_back(listener);
std::cout << "Added a method instance for DataType::Type_2" << "\n";
}
else {
std::cout << "Listener type not supported" << "\n";
}
}
void CallAllListnersWhohaveSuscribed() {
if (!my_data_listeners_1.empty()) {
std::string send_message_1 = "some message 123";
std::pair <DataType, std::string> info_to_send_1 = std::make_pair (DataType::Type_1, send_message_1);
for(auto const &listener : my_data_listeners_1) {
listener(info_to_send_1);
}
}
if (!my_data_listeners_2.empty()) {
std::string send_message_2 = "some message 456";
std::pair <DataType, std::string> info_to_send_2 = std::make_pair (DataType::Type_2, send_message_2);
for(auto const &listener : my_data_listeners_2) {
listener(info_to_send_2);
}
}
}
int main() {
// Add ListenerMethod_Instance_1 for instance
DataType data_type_1 = DataType::Type_1;
auto listener_instance_1 = std::bind(ListenerMethod_Instance_1, std::placeholders::_1);
AddListener(listener_instance_1, data_type_1);
// Add ListenerMethod_Instance_2 for instance
DataType data_type_2 = DataType::Type_2;
auto listener_instance_2 = std::bind(ListenerMethod_Instance_2, std::placeholders::_1);
AddListener(listener_instance_2, data_type_2);
CallAllListnersWhohaveSuscribed();
return 0;
}
Following is the output of the program:
./stdFunctionTest
Added a method instance for DataType::Type_1
Added a method instance for DataType::Type_2
ListenerMethod_Instance_1 called with message some message 123
ListenerMethod_Instance_2 called with message some message 456
But here is how I want to modify & struggling with. The caveat is that every ListenerMethod_Instance_1 & ListenerMethod_Instance_2 have to parse the pair to get their info which I don't want to. I want to enable a method of any C++ primitive data type be it "int, bool, float or double" to be able to be added into the listeners vector & receive the callback. For example following method should be "add-able" into AddListener.
void ListenerMethod_Instance_3(int integer_data) {
std::cout << "ListenerMethod_Instance_3 called with integer_data " << integer_data << "\n";
}
Looking at this link here looks somewhat possible someway. But I'm struggling to adapt it to my use-case here. Please suggest.
So, basically how can I achieve templates functionality with std::functions ?
struct anything_view_t {
void* ptr=0;
template<class T, std::enable_if_t<!std::is_same<anything_view_t, std::decay_t<T>>{}, int> =0>
anything_view_t(T&&t):ptr(std::addressof(t)){}
anything_view_t()=default;
anything_view_t(anything_view_t const&)=default;
anything_view_t& operator=(anything_view_t const&)=default;
template<class T>
operator T() const { return *static_cast<T*>(ptr); }
};
this is a very unsafe type erasing view of anything.
struct any_callbacks {
std::unordered_map<std::type_index, std::vector<std::function<void(anything_view_t)>>> table;
template<class T>
void add_callback( std::function<void(T)> f ){
table[typeid(T)].push_back(f);
}
template<class T>
void invoke_callbacks(T t) const {
auto it = table.find(typeid(T));
if (it==table.end()) return;
for(auto&&f:it->second)
f(t);
}
};
something like the above should work. The type T must match exactly. References not supported. Code not compiled, design is sound, probably has typos.
This is not restructed to primitive types. You should pass T explicitly, don't rely on deduction as that is fragile.

How to work around C++ pointer-to-member function limitation

C++ has limited ability to use pointer-to-member functions. I need something that will allow me to dynamically choose a callback member function, in order to use the Visitor pattern of the XMLNode::Accept(XMLVisitor *visitor) method from the TinyXML2 library.
To use XMLNode::Accept(), I must call it with a class which implements the XMLVisitor interface. Hence:
typedef bool (*Callback)(string, string);
class MyVisitor : public tinyxml2::XMLVisitor {
public:
bool VisitExit(const tinyxml2::XMLElement &e) {
callback(e.Name(), e.GetText());
}
Callback callback;
}
This works fine if my caller is NOT an object which wants to use one of its own methods as a callback function (so that it can access class variables). For example, this works:
bool myCallBackFunc(string e, string v) {
cout << "Element " << e << " has value " << v << endl;
return true;
}
int main(...) {
tinyxml2::XMLDocument doc;
doc.LoadFile("somefile.xml");
MyVisitor visit;
visit.callback = myCallBackFunc;
doc.Accept(&visit);
}
However, in my use case, the parsing is done inside a method in a class. I have multiple applications which have similar but unique such classes. I'd like to use only one generic MyVisitor class, rather than have the visitor class have unique knowledge of the internals of each class which will call it.
Thus, it would be convenient if the callback function were a method in each calling class so that I can affect the internal state of the object instantiated from that calling class.
Top level: I have 5 server applications which talk to 5 different trading partners, who all send XML responses, but each is enough different that each server app has a class which is unique to that trading partner. I'm trying to follow good OO and DRY design, and avoid extra classes having unique knowledge while still doing basically the same work.
Here's the class method I want Accept() to call back.
ServiceClass::changeState(string elem, string value) {
// Logic which sets member vars based on element found and its value.
}
Here's the class method which will call Accept() to walk the XML:
ServiceClass::processResponse(string xml) {
// Parse XML and do something only if certain elements present.
tinyxml2::XMLDocument doc;
doc.Parse(xml.c_str(), xml.length());
MyVisitor visit;
visit.callback = &changeState; // ERROR. Does not work.
visit.callback = &ServiceClass::changeState; // ERROR. Does not work.
doc.Accept(&visit);
}
What's a simple way to get what I want? I can imagine more classes with derived classes unique to each situation, but that seems extremely verbose and clumsy.
Note: In the interest of brevity, my sample code above has no error checking, no null checking and may even have minor errors (e.g. treating const char * as a string ;-).
Below is the std::bind(..) example for what you're trying to do in C++11. For earlier C++ versions you could use the boost::bind utilities.
Fix your MyVisitor::VisitExit(...) method to return a boolean, by the way.
The code is converting const char * to std::string. tinyxml2 does not guarantee that the char * arguments from Name() or GetText() are not null. In fact in my experience they will be null at some point. You should guard against this. For the sake of not modifying your example too much I've not protected against this possibility everywhere in the example.
typedef bool(*Callback)(string, string);
using namespace std;
class MyVisitor : public tinyxml2::XMLVisitor {
public:
bool VisitExit(const tinyxml2::XMLElement &e) {
// return callback(e.Name(), e.GetText());
return true;
}
Callback callback;
};
/** Typedef to hopefully save on confusing syntax later */
typedef std::function< bool(const char * element_name, const char * element_text) > visitor_fn;
class MyBoundVisitor : public tinyxml2::XMLVisitor {
public:
MyBoundVisitor(visitor_fn fn) : callback(fn) {}
bool VisitExit(const tinyxml2::XMLElement &e) {
return callback(e.Name() == nullptr ? "\0" : e.Name(), e.GetText() == nullptr ? "\0": e.GetText());
}
visitor_fn callback;
};
bool
myCallBackFunc(string e, string v) {
cout << "Element " << e << " has value " << v << endl;
return true;
}
int
main()
{
tinyxml2::XMLDocument doc;
doc.LoadFile("somefile.xml");
MyVisitor visit;
visit.callback = myCallBackFunc;
doc.Accept(&visit);
visitor_fn fn = myCallBackFunc; // copy your function pointer into the std::function<> type
MyBoundVisitor visit2(fn); // note: declare this outside the Accept(..) , do not use a temporary
doc.Accept(&visit2);
}
So from within the ServiceClass method you'd do:
ServiceClass::processResponse(string xml) {
// Parse XML and do something only if certain elements present.
tinyxml2::XMLDocument doc;
doc.Parse(xml.c_str(), xml.length());
// presuming changeState(const char *, const char *) here
visitor_fn fn = std::bind(&ServiceClass::changeState,this,std::placeholders::_1,std::placeholders::_2);
MyBoundVisitor visit2(fn); // the method pointer is in the fn argument, together with the instance (*this) it is a method for.
doc.Accept(&visit);
}
You can use generics in order to support whichever callback you'd like.
I've tried to mock the classes of the library in order to give you a fully runnable example:
#include <string>
#include <iostream>
#include <functional>
class XmlNode {
public:
XmlNode(const std::string& n, const std::string t) : name(n), txt(t) {}
const std::string& Name() const { return name; }
const std::string& GetText() const { return txt; }
private:
std::string name;
std::string txt;
};
class XMLVisitor {
public:
virtual void VisitExit(const XmlNode& node) = 0;
virtual ~XMLVisitor() {}
};
template<typename T>
class MyVisitor : XMLVisitor {
public:
MyVisitor() {}
void myInnerPrint(const XmlNode& node) {
std::cout << "MyVisitor::myInnerPrint" << std::endl;
std::cout << "node.Name(): " << node.Name() << std::endl;
std::cout << "node.GetText(): " << node.GetText() << std::endl;
}
void SetCallback(T newCallback) {
callback = newCallback;
}
virtual void VisitExit(const XmlNode& node) {
callback(node);
}
T callback;
};
int main() {
XmlNode node("In", "Member");
MyVisitor<std::function<void(const XmlNode&)>> myVisitor;
auto boundCall =
[&myVisitor](const XmlNode& node) -> void {
myVisitor.myInnerPrint(node);
};
myVisitor.SetCallback(boundCall);
myVisitor.VisitExit(node);
return 0;
}
First define a template and a helper function:
namespace detail {
template<typename F>
struct xml_visitor : tinyxml2::XMLVisitor {
xml_visitor(F&& f) : f_(std::move(f)) {}
virtual void VisitExit(const tinyxml2::XMLElement &e) {
f_(e);
}
private:
F f_;
};
}
template<class F>
auto make_xml_visitor(F&& f)
{
return detail::xml_visitor<std::decay_t<F>>(std::forward<F>(f));
}
Then use the helper function to construct a custom visitor from a lambda which captures this:
void ServiceClass::processResponse(std::string xml) {
// Parse XML and do something only if certain elements present.
tinyxml2::XMLDocument doc;
doc.Parse(xml.c_str(), xml.length());
auto visit = make_xml_visitor([this](const auto& elem)
{
this->changeState(elem.Name(), elem.GetText);
});
doc.Accept(std::addressof(visit));
}
The rule is that a function pointer must always accept a void * which is passed in to the module which calls it, and passed back. Or use a lambda which is the same thing with some of the machinery automated for you. (The void * is the "closure").
So
typedef bool (*Callback)(string, string, void *context);
class MyVisitor : public tinyxml2::XMLVisitor {
public:
bool VisitExit(const tinyxml2::XMLElement &e) {
callback(e.Name(), e.GetText(), contextptr);
}
Callback callback;
void *contextptr;
}
bool myCallBackFunc(string e, string v, void *context) {
ServiceClass *service = (ServiceClass *) context;
cout << "Element " << e << " has value " << v << endl;
service->ChangeState(e, v);
return true;
}

C++ random access iterators for containers with elements loaded on demand

I'm currently working on a small project which requires loading messages from a file. The messages are stored sequentially in the file and files can become huge, so loading the entire file content into memory is unrewarding.
Therefore we decided to implement a FileReader class that is capable of moving to specific elements in the file quickly and load them on request. Commonly used something along the following lines
SpecificMessage m;
FileReader fr;
fr.open("file.bin");
fr.moveTo(120); // Move to Message #120
fr.read(&m); // Try deserializing as SpecificMessage
The FileReader per se works great. Therefore we thought about adding STL compliant iterator support as well: A random access iterator that provides read-only references to specific messages. Used in the following way
for (auto iter = fr.begin<SpecificMessage>(); iter != fr.end<SpecificMessage>(); ++iter) {
// ...
}
Remark: the above assumes that the file only contains messages of type SpecificMessage. We've been using boost::iterator_facade to simplify the implementation.
Now my question boils down to: how to implement the iterator correctly? Since FileReader does not actually hold a sequence of messages internally, but loads them on request.
What we've tried so far:
Storing the message as an iterator member
This approach stores the message in the iterator instance. Which works great for simple use-cases but fails for more complex uses. E.g. std::reverse_iterator has a dereference operation that looks like this
reference operator*() const
{ // return designated value
_RanIt _Tmp = current;
return (*--_Tmp);
}
This breaks our approach as a reference to a message from a temporary iterator is returned.
Making the reference type equal the value type
#DDrmmr in the comments suggested making the reference type equal the value type, so that a copy of the internally stored object is returned. However, I think this is not valid for the reverse iterator which implements the -> operator as
pointer operator->() const {
return (&**this);
}
which derefs itself, calls the *operator which then returns a copy of a temporary and finally returns the address of this temporary.
Storing the message externally
Alternatively I though about storing the message externally:
SpecificMessage m;
auto iter = fr.begin<SpecificMessage>(&m);
// ...
which also seems to be flawed for
auto iter2 = iter + 2
which will have both iter2 and iter point to the same content.
As I hinted in my other answer, you could consider using memory mapped files. In the comment you asked:
As far as memory mapped files is concerned, this seems not what I want to have, as how would you provide an iterator over SpecificMessages for them?
Well, if your SpecificMessage is a POD type, you could just iterate over the raw memory directly. If not, you could have a deserialization helper (as you already have) and use Boost transform_iterator to do the deserialization on demand.
Note that we can make the memory mapped file managed, effectively meaning that you can just use it as a regular heap, and you can store all standard containers. This includes node-based containers (map<>, e.g.), dynamic-size containers (e.g. vector<>) in addition to the fixed-size containers (array<>) - and any combinations of those.
Here's a demo that takes a simple SpecificMessage that contains a string, and (de)derializes it directly into shared memory:
using blob_t = shm::vector<uint8_t>;
using shared_blobs = shm::vector<blob_t>;
The part that interests you would be the consuming part:
bip::managed_mapped_file mmf(bip::open_only, DBASE_FNAME);
shared_blobs* table = mmf.find_or_construct<shared_blobs>("blob_table")(mmf.get_segment_manager());
using It = boost::transform_iterator<LazyLoader<SpecificMessage>, shared_blobs::const_reverse_iterator>;
// for fun, let's reverse the blobs
for (It first(table->rbegin()), last(table->rend()); first < last; first+=13)
std::cout << "blob: '" << first->contents << "'\n";
// any kind of random access is okay, though:
auto random = rand() % table->size();
SpecificMessage msg;
load(table->at(random), msg);
std::cout << "Random blob #" << random << ": '" << msg.contents << "'\n";
So this prints each 13th message, in reverse order, followed by a random blob.
Full Demo
The sample online uses the lines of the sources as "messages".
Live On Coliru
#include <boost/interprocess/file_mapping.hpp>
#include <boost/interprocess/managed_mapped_file.hpp>
#include <boost/container/scoped_allocator.hpp>
#include <boost/interprocess/containers/vector.hpp>
#include <iostream>
#include <boost/iterator/transform_iterator.hpp>
#include <boost/range/iterator_range.hpp>
static char const* DBASE_FNAME = "database.map";
namespace bip = boost::interprocess;
namespace shm {
using segment_manager = bip::managed_mapped_file::segment_manager;
template <typename T> using allocator = boost::container::scoped_allocator_adaptor<bip::allocator<T, segment_manager> >;
template <typename T> using vector = bip::vector<T, allocator<T> >;
}
using blob_t = shm::vector<uint8_t>;
using shared_blobs = shm::vector<blob_t>;
struct SpecificMessage {
// for demonstration purposes, just a string; could be anything serialized
std::string contents;
// trivial save/load serialization code:
template <typename Blob>
friend bool save(Blob& blob, SpecificMessage const& msg) {
blob.assign(msg.contents.begin(), msg.contents.end());
return true;
}
template <typename Blob>
friend bool load(Blob const& blob, SpecificMessage& msg) {
msg.contents.assign(blob.begin(), blob.end());
return true;
}
};
template <typename Message> struct LazyLoader {
using type = Message;
Message operator()(blob_t const& blob) const {
Message result;
if (!load(blob, result)) throw std::bad_cast(); // TODO custom excepion
return result;
}
};
///////
// for demo, create some database contents
void create_database_file() {
bip::file_mapping::remove(DBASE_FNAME);
bip::managed_mapped_file mmf(bip::open_or_create, DBASE_FNAME, 1ul<<20); // Even sparse file size is limited on Coliru
shared_blobs* table = mmf.find_or_construct<shared_blobs>("blob_table")(mmf.get_segment_manager());
std::ifstream ifs("main.cpp");
std::string line;
while (std::getline(ifs, line)) {
table->emplace_back();
save(table->back(), SpecificMessage { line });
}
std::cout << "Created blob table consisting of " << table->size() << " blobs\n";
}
///////
void display_random_messages() {
bip::managed_mapped_file mmf(bip::open_only, DBASE_FNAME);
shared_blobs* table = mmf.find_or_construct<shared_blobs>("blob_table")(mmf.get_segment_manager());
using It = boost::transform_iterator<LazyLoader<SpecificMessage>, shared_blobs::const_reverse_iterator>;
// for fun, let's reverse the blobs
for (It first(table->rbegin()), last(table->rend()); first < last; first+=13)
std::cout << "blob: '" << first->contents << "'\n";
// any kind of random access is okay, though:
auto random = rand() % table->size();
SpecificMessage msg;
load(table->at(random), msg);
std::cout << "Random blob #" << random << ": '" << msg.contents << "'\n";
}
int main()
{
#ifndef CONSUMER_ONLY
create_database_file();
#endif
srand(time(NULL));
display_random_messages();
}
You are having issues because your iterator does not conform to the forward iterator requirements. Specifically:
*i must be an lvalue reference to value_type or const value_type ([forward.iterators]/1.3)
*i cannot be a reference to an object stored in the iterator itself, due to the requirement that two iterators are equal if and only if they are bound to the same object ([forward.iterators]/6)
Yes, these requirements are a huge pain in the butt, and yes, that means that things like std::vector<bool>::iterator are not random access iterators even though some standard library implementations incorrectly claim that they are.
EDIT: The following suggested solution is horribly broken, in that dereferencing a temporary iterator returns a reference to an object that may not live until the reference is used. For example, after auto& foo = *(i + 1); the object referenced by foo may have been released. The implementation of reverse_iterator referenced in the OP will cause the same problem.
I'd suggest that you split your design into two classes: FileCache that holds the file resources and a cache of loaded messages, and FileCache::iterator that holds a message number and lazily retrieves it from the FileCache when dereferenced. The implementation could be something as simple as storing a container of weak_ptr<Message> in FileCache and a shared_ptr<Message> in the iterator: Simple demo
I have to admit I may not fully understand the trouble you have with holding the current MESSAGE as a member of Iter. I would associate each iterator with the FileReader it should read from and implement it as a lightweight encapsulation of a read index for FileReader::(read|moveTo). The most important method to overwtite is boost::iterator_facade<...>::advance(...) which modifies the current index and tries to pull a new MESSAGE from the FileReader If this fails it flags the the iterator as invalid and dereferencing will fail.
template<class MESSAGE,int STEP>
class message_iterator;
template<class MESSAGE>
class FileReader {
public:
typedef message_iterator<MESSAGE, 1> const_iterator;
typedef message_iterator<MESSAGE,-1> const_reverse_iterator;
FileReader();
bool open(const std::string & rName);
bool moveTo(int n);
bool read(MESSAGE &m);
// get the total count of messages in the file
// helps us to find end() and rbegin()
int getMessageCount();
const_iterator begin() {
return const_iterator(this,0);
}
const_iterator end() {
return const_iterator(this,getMessageCount());
}
const_reverse_iterator rbegin() {
return const_reverse_iterator(this,getMessageCount()-1);
}
const_reverse_iterator rend() {
return const_reverse_iterator(this,-1);
}
};
// declaration of message_iterator moving over MESSAGE
// STEP is used to specify STEP size and direction (e.g -1 == reverse)
template<class MESSAGE,int STEP=1>
class message_iterator
: public boost::iterator_facade<
message_iterator<MESSAGE>
, const MESSAGE
, boost::random_access_traversal_tag
>
{
typedef boost::iterator_facade<
message_iterator<MESSAGE>
, const MESSAGE
, boost::random_access_traversal_tag
> super;
public:
// constructor associates an iterator with its FileReader and a given position
explicit message_iterator(FileReader<MESSAGE> * p=NULL,int n=0): _filereader(p),_idx(n),_valid(false) {
advance(0);
}
bool equal(const message_iterator & i) const {
return i._filereader == _filereader && i._idx == _idx;
}
void increment() {
advance(+1);
}
void decrement() {
advance(-1);
}
// overwrite with central functionality. Move to a given relative
// postion and check wether the position can be read. If move/read
// fails we flag the iterator as incalid.
void advance(int n) {
_idx += n*STEP;
if(_filereader!=NULL) {
if( _filereader->moveTo( _idx ) && _filereader->read(_m)) {
_valid = true;
return;
}
}
_valid = false;
}
// Return a ref to the currently cached MESSAGE. Throw
// an acception if positioning at this location in advance(...) failes.
typename super::reference dereference() const {
if(!_valid) {
throw std::runtime_error("access to invalid pos");
}
return _m;
}
private:
FileReader<MESSAGE> * _filereader;
int _idx;
bool _valid;
MESSAGE _m;
};
Boost PropertyMap
You could avoid writing the bulk of the code using Boost PropertyMap:
Live On Coliru
#include <boost/property_map/property_map.hpp>
#include <boost/property_map/function_property_map.hpp>
using namespace boost;
struct SpecificMessage {
// add some data
int index; // just for demo
};
template <typename Message>
struct MyLazyReader {
typedef Message type;
std::string fname;
MyLazyReader(std::string fname) : fname(fname) {}
Message operator()(size_t index) const {
Message m;
// FileReader fr;
// fr.open(fname);
// fr.moveTo(index); // Move to Message
// fr.read(&m); // Try deserializing as SpecificMessage
m.index = index; // just for demo
return m;
}
};
#include <iostream>
int main() {
auto lazy_access = make_function_property_map<size_t>(MyLazyReader<SpecificMessage>("file.bin"));
for (int i=0; i<10; ++i)
std::cout << lazy_access[rand()%256].index << "\n";
}
Sample output is
103
198
105
115
81
255
74
236
41
205
Using Memory Mapped Files
You could store a map of index -> BLOB objects in a shared vector<array<byte, N>>, flat_map<size_t, std::vector<uint8_t> > or similar.
So, now you only have to deserialize from myshared_map[index].data() (begin() and end() in case the BLOB size varies)

How to iterate through a collection of instances of a class and access its template member

So, I have got the following classes and methods:
Property: Has a single member of type int (named mTag)
TypedProperty: Inherits from the Property class and adds a member called mValue of type T to it.
PropertyList: A class which Maintains a std::set of Property and has an Add and Print method.
CheckSubset: A method which checks if a std::set is included in another set.
I don't know how I should implement the CheckSubset method. Because I do not know how to iterate through a set<Property> and access to the template member (mValue). I also tried to use the includes method, which did not work (even if it worked, I would have no idea how it did!). The same problem exists in the PropertyList::Print method, where I do not know what cast should be used.
Any advice on the implementation of CheckSubset and Print methods would be appreciated!
Updated source code (using pointer)
#include <string>
#include <iostream>
#include <set>
#include <algorithm>
#include <tr1/memory>
using namespace std;
/////////////////// Property Class //////////////////////
class Property
{
public:
Property(){};
Property(const int tag)
: mTag(tag) {}
virtual ~Property() {}
int mTag;
bool operator<(const Property &property) const
{
return mTag < property.mTag;
}
};
/////////////////// TypedProperty Class /////////////////
template< typename T >
class TypedProperty : public Property
{
public:
TypedProperty (const int tag, const T& value)
: Property(tag), mValue(value){}
T mValue;
};
/////////////////////////////////////////////////////////
typedef std::tr1::shared_ptr<Property> PropertyPtr;
/////////////////// PropertyList Class /////////////////
class PropertyList
{
public:
PropertyList(){};
virtual ~PropertyList(){};
template <class T>
void Add(int tag, T value)
{
PropertyPtr ptr(new TypedProperty<T>(tag, value));
mProperties.insert(ptr);
}
void Print()
{
for(set<PropertyPtr>::iterator itr = mProperties.begin(); itr != mProperties.end(); itr++)
{
cout << ((PropertyPtr)*itr)->mTag << endl;
// What should I do to print mValue? I do not know its type
// what should *itr be cast to?
}
}
set<PropertyPtr> mProperties;
};
//////////////////// Check Subset ///////////////////////
/*
* Checks if subset is included in superset
*/
bool CheckSubset(set<PropertyPtr> &superset, set<PropertyPtr> &subset)
{
// How can I iterate over superset and subset values while I do not know
// the type of mValue inside each Property?
// I also tried the following method which does not seem to work correctly
return includes(superset.begin(), superset.end(),
subset.begin(), subset.end());
}
int main()
{
PropertyList properties1;
properties1.Add(1, "hello");
properties1.Add(2, 12);
properties1.Add(3, 34);
properties1.Add(4, "bye");
properties1.Print();
PropertyList properties2;
properties2.Add(1, "hello");
properties2.Add(3, 34);
if(CheckSubset(properties1.mProperties, properties2.mProperties)) // should be true
cout << "properties2 is subset!" << endl;
PropertyList properties3;
properties3.Add(1, "hello");
properties3.Add(4, 1234);
if(CheckSubset(properties1.mProperties, properties3.mProperties)) // should be false
cout << "properties3 is subset!" << endl;
}
What you want, cannot be done with the current design.
Your approach fails with std::set<Property>.
std::set<Property> will slice. That means that it will only copy the Property part and forget to copy the additional TypedProperty<T> members.
As a result, inside PropertyList::print(), there is no way to access the mValue.
If you want to store TypedProperty<T>s inside a std::set, you must use some sort of pointer. I.e. either std::set<Property*>, or a smart pointer version.
For solving the problem in Print method of PropertyList, you could write a Print method for TypedProperty class, which prints its tag and value.
But about the problem in accessing mValue which you want to do some operations on, I can't think of a way using normal types and templates to get the mValue without engaging your parent class Property with template type of TypedProperty (which seems undesirable). But you could get the address of mValue and cast it to void* to eliminate the type problem. This way you will face another problem, that you can not point to value of a void* pointer, so you can not work with your pointer in parent level. Therefore, you should write a method (implemented by TypedProperty) that takes a void* pointer and casts it to the type defined in child and perform the desired operation.
For example in the following code, I assumed you want to check equality of a value in a TypedProperty with another one of the same type (IsEqual method).
Now you can implement simply CheckSubset using IsEqual (checking two elements would be like: superItr->IsEqual(subItr->GetValue())).
class Property
{
public:
Property(){};
Property(const int tag)
: mTag(tag) {}
virtual ~Property() {}
virtual void* GetValue() = 0;
virtual bool IsEqual(void* value) = 0;
virtual void Print() = 0;
int mTag;
bool operator<(const Property &property) const
{
return mTag < property.mTag;
}
};
template< typename T >
class TypedProperty : public Property
{
public:
TypedProperty (const int tag, const T& value)
: Property(tag), mValue(value){}
void* GetValue()
{
return &mValue;
}
bool IsEqual(void* value)
{
return *((T*)value) == mValue;
}
void Print()
{
cout << "Tag: " << mTag << ", Value: " << mValue << endl;
}
T mValue;
};
typedef std::tr1::shared_ptr<Property> PropertyPtr;
class PropertyList
{
public:
PropertyList(){};
virtual ~PropertyList(){};
template <class T>
void Add(int tag, T value)
{
PropertyPtr ptr(new TypedProperty<T>(tag, value));
mProperties.insert(ptr);
}
void Print()
{
cout << "-----------" << endl;
for(set<PropertyPtr>::iterator itr = mProperties.begin(); itr != mProperties.end(); itr++)
{
(*itr)->Print();
}
}
set<PropertyPtr> mProperties;
};