How do variant and any from the boost library work internally? In a project I am working on, I currently use a tagged union. I want to use something else, because unions in C++ don't let you use objects with constructors, destructors or overloaded assignment operators.
I queried the size of any and variant, and did some experiments with them. In my platform, variant takes the size of its longest possible type plus 8 bytes: I think it my just be 8 bytes o type information and the rest being the stored value. On the other hand, any just takes 8 bytes. Since i'm on a 64-bit platform, I guess any just holds a pointer.
How does Any know what type it holds? How does Variant achieve what it does through templates? I would like to know more about these classes before using them.
If you read the boost::any documentation they provide the source for the idea: http://www.two-sdg.demon.co.uk/curbralan/papers/ValuedConversions.pdf
It's basic information hiding, an essential C++ skill to have. Learn it!
Since the highest voted answer here is totally incorrect, and I have my doubts that people will actually go look at the source to verify that fact, here's a basic implementation of an any like interface that will wrap any type with an f() function and allow it to be called:
struct f_any
{
f_any() : ptr() {}
~f_any() { delete ptr; }
bool valid() const { return ptr != 0; }
void f() { assert(ptr); ptr->f(); }
struct placeholder
{
virtual ~placeholder() {}
virtual void f() const = 0;
};
template < typename T >
struct impl : placeholder
{
impl(T const& t) : val(t) {}
void f() const { val.f(); }
T val;
};
// ptr can now point to the entire family of
// struct types generated from impl<T>
placeholder * ptr;
template < typename T >
f_any(T const& t) : ptr(new impl<T>(t)) {}
// assignment, etc...
};
boost::any does the same basic thing except that f() actually returns typeinfo const& and provides other information access to the any_cast function to work.
The key difference between boost::any and boost::variant is that any can store any type, while variant can store only one of a set of enumerated types. The any type stores a void* pointer to the object, as well as a typeinfo object to remember the underlying type and enforce some degree of type safety. In boost::variant, it computes the maximum sized object, and uses "placement new" to allocate the object within this buffer. It also stores the type or the type index.
Note that if you have Boost installed, you should be able to see the source files in "any.hpp" and "variant.hpp". Just search for "include/boost/variant.hpp" and "include/boost/any.hpp" in "/usr", "/usr/local", and "/opt/local" until you find the installed headers, and you can take a look.
Edit
As has been pointed out in the comments below, there was a slight inaccuracy in my description of boost::any. While it can be implemented using void* (and a templated destroy callback to properly delete the pointer), the actualy implementation uses any<T>::placeholder*, with any<T>::holder<T> as subclasses of any<T>::placeholder for unifying the type.
boost::any just snapshots the typeinfo while the templated constructor runs: it has a pointer to a non-templated base class that provides access to the typeinfo, and the constructor derived a type-specific class satisfying that interface. The same technique can actually be used to capture other common capabilities of a set of types (e.g. streaming, common operators, specific functions), though boost doesn't offer control of this.
boost::variant is conceptually similar to what you've done before, but by not literally using a union and instead taking a manual approach to placement construction/destruction of objects in its buffer (while handling alignment issues explicitly) it works around the restrictions that C++ has re complex types in actual unions.
Related
Often when writing templated code, I find myself needing to store an instance of the template type in a member variable. For example, I might need to cache a value to be used later on. I would like to be able to write my code as:
struct Foo
{
template<typename T>
T member;
template<typename T>
void setMember(T value)
{
member<T> = value;
}
template<typename T>
T getMember()
{
return member<T>;
}
};
Where members are specialized as they are used. My question:
Is such templated member variable possible with current C++ generative coding facilities?
If not, are there any proposals for such a language feature?
If not, are there any technical reasons why such a thing is not possible?
It should be obvious that I do not want to list all possible types (e.g. in a std::variant) as that is not generative programming and would not be possible if the user of the library is not the same as the author.
Edit: I think this somewhat answers my 3rd question from above. The reason being that today's compilers are not able to postpone instantiation of objects to after the whole program has been parsed:
https://stackoverflow.com/a/27709454/3847255
This is possible in the library by combining existing facilities.
The simplest implementation would be
std::unordered_map<std::type_index, std::any>
This is mildly inefficient since it stores each std::type_index object twice (once in the key and once inside each std::any), so a std::unordered_set<std::any> with custom transparent hash and comparator would be more efficient; this would be more work though.
Example.
As you say, the user of the library may not be the same as the author; in particular, the destructor of Foo does not know which types were set, but it must locate those objects and call their destructors, noting that the set of types used may be different between instances of Foo, so this information must be stored in a runtime container within Foo.
If you're wary about the RTTI overhead implied by std::type_index and std::any, we can replace them with lower-level equivalents. For std::type_index you can use a pointer to a static tag variable template instantiation (or any similar facility), and for std::any you can use a type-erased std::unique_ptr<void, void(*)(void*)> where the deleter is a function pointer:
using ErasedPtr = std::unique_ptr<void, void(*)(void*)>;
std::unordered_map<void*, ErasedPtr> member;
struct tag {};
template<class T> inline static tag type_tag;
member.insert_or_assign(&type_tag<T>, ErasedPtr{new T(value), [](void* p) {
delete static_cast<T*>(p);
}});
Example. Note that once you make the deleter of std::unique_ptr a function pointer, it is no longer default-constructible, so we can't use operator[] any more but must use insert_or_assign and find. (Again, we've got the same DRY violation / inefficiency, since the deleter could be used as the key into the map; exploiting this is left as an exercise for the reader.)
Is such templated member variable possible with current C++ generative coding facilities?
No, not exactly what you describe. What is possible is to make the enclosing class a template and use the template parameters to describe the types of the class' members.
template< typename T >
struct Foo
{
T member;
void setMember(T value)
{
member = value;
}
T getMember()
{
return member;
}
};
In C++14 and later, there are variable templates, but you can't make a template non-static data member of a class.
If not, are there any proposals for such a language feature?
Not that I know of.
If not, are there any technical reasons why such a thing is not possible?
The primary reason is that that would make it impossible to define binary representation of the class. As opposed to templates, a class is a type, which means its representation must be fixed, meaning that at any place in the program Foo and Foo::member must mean the same things - same types, same object sizes and binary layout, and so on. A template, on the other hand, is not a type (or, in case of variable templates, is not an object). It becomes one when it is instantiated, and each template instantiation is a separate type (in case of variable templates - object).
I created a vector that stores any type of value (int, bool, string, &, object,...). I can store stuff, but i have no idea how to get an specific element ([index]). EDIT: (without adding the type)
I haven't tried many things because i have no clue. The only tecnic i can think of is dynamic_cast, but since is a template makes no sense.
#include <vector>
#include <iostream>
struct slot {
virtual void print() = 0;
};
template<typename kind>
struct item : public slot {
kind value;
item(kind value) : value{ value } {};
void print()override { std::cout << value<<std::endl; }
};
class Bag {
std::vector<slot*> backpack;
public:
template<typename kind>
void append(kind stuff) {
backpack.push_back(new item<kind>(stuff));
}
void print_to_test() {
for (slot* it : backpack) { it->print(); }
}
//kind get()...How do i get an item value?
};
Bag bag;
bag.append(1);
bag.append(true);
bag.append("Hola");
bag.append(1232131);
void* a = nullptr;
bag.append(a);
bag.print_to_test();
//works fine, prints everything
//but can't get an specific value, like bag[index]
The dynamic cast will work. Note that it requires at least one virtual method - a destructor can be used if nothing else.
Yes, there isn't a way how can you reliably retrieve the value. C++ doesn't have a general object base as other languages nor any useful runtime reflection.
What you've implemented is known as type erasure and is actually used in e.g. std::function or std::any.
You should not use raw pointers and new for ownership. Use std::unique_ptr instead. Also since you only refer to the objects through the base class you need the virtual destructor. Otherwise, you won't be able to delete them correctly, again one more reason for std::unique_ptr as it deletes the pointer by itself.
If you want to get the value, what would you do with it? To quote Bjarne Stroustrup:
There is no useful universal class: a truly universal carries no semantics of its own.
Yes, you can store this value in std::any, but that's it, what would you want to do with it? Print it? Well, in that case it's no longer any but printable as in your example, so either use templates or virtual methods to express these traits in the types of the stored objects.
Consider using std::variant for known set of possible types and prefer std::any to owning void*. If you need the latter rethink your design choices first.
As others pointed out, dynamic_cast will work fine (as long as the caller knows what type is in the specific index) :
item<int>& someInt = dynamic_cast<item<int>&>(*bag.backpack[0]);
std::cout << "someInt: ";
someInt.print();
Note: You didn't provide accessor to backpack member, so I assumed it's public
Output:
someInt: 1
I have a limited set of very different types, from which I want to store instances in a single collection, specifically a map. To this end, I use the type erasure idiom, ie. I have a non-templated base class from which the templated, type specific class inherits:
struct concept
{
virtual std::unique_ptr<concept> copy() = 0; // example member function
};
template <typename T>
struct model : concept
{
T value;
std::unique_ptr<concept> copy() override { ... }
}
I then store unique_ptrs to concept in my map. To retrieve the value, I have a templated function which does a dynamic cast to the specified type.
template <typename T>
void get(concept& c, T& out) {
auto model = dynamic_cast<model<T>>(&c);
if (model == nullptr) throw "error, wrong type";
out = model->value;
}
What I don't like about this solution is, that specifying a wrong T is only detected at runtime. I'd really really like this to be done at compile time.
My options are as I see the following, but I don't think they can help here:
Using ad hoc polymorphism by specifying free functions with each type as an overload, or a template function, but I do not know where to store the result.
Using CRTP won't work, because then the base class would need to be templated.
Conceptually I would need a virtual function which takes an instance of a class where the result will be stored. However since my types are fundamentally different, this class would need to be templated, which does not work with virtual.
Anyways, I'm not even sure if this is logically possible, but I would be very glad if there was a way to do this.
For a limited set of types, your best option is variant. You can operate on a variant most easily by specifying what action you would take for every single variant, and then it can operate on a variant correctly. Something along these lines:
std::unordered_map<std::string, std::variant<Foo, Bar>> m;
m["a_foo"] = Foo{};
m["a_bar"] = Bar{};
for (auto& e : m) {
std::visit(overloaded([] (Foo&) { std::cerr << "a foo\n"; }
[] (Bar&) { std::cerr << "a bar\n"; },
e.second);
}
std::variant is c++17 but is often available in the experimental namespace beforehand, you can also use the version from boost. See here for the definition of overloaded: http://en.cppreference.com/w/cpp/utility/variant/visit (just a small utility the standard library unfortunately doesn't provide).
Of course, if you are expecting that a certain key maps to a particular type, and want to throw an error if it doesn't, well, there is no way to handle that at compile time still. But this does let you write visitors that do the thing you want for each type in the variant, similar to a virtual in a sense but without needing to actually have a common interface or base class.
You cannot do compile-time type checking for an erased type. That goes against the whole point of type erasure in the first place.
However, you can get an equivalent level of safety by providing an invariant guarantee that the erased type will match the expected type.
Obviously, wether that's feasible or not depends on your design at a higher level.
Here's an example:
class concept {
public:
virtual ~concept() {}
};
template<typename T>
struct model : public concept {
T value;
};
class Holder {
public:
template<typename T>
void addModel() {
map.emplace(std::type_index(typeid(T)), std::make_unique<model<T><());
}
template<typename T>
T getValue() {
auto found = types.find(std::type_index(typeid(T)));
if(found == types.end()) {
throw std::runtime_error("type not found");
}
// no need to dynamic cast here. The invariant is covering us.
return static_cast<model<T>*>(found->second.get())->value;
}
private:
// invariant: map[type] is always a model<type>
std::map<std::type_index, std::unique_ptr<concept>> types;
};
The strong encapsulation here provides a level of safety almost equivalent to a compile-time check, since map insertions are aggressively protected to maintain the invariant.
Again, this might not work with your design, but it's a way of handling that situation.
Your runtime check occurs at the point where you exit type erasure.
If you want to compile time check the operation, move it within the type erased boundaries, or export enough information to type erase later.
So enumerate the types, like std variant. Or enumerate the algorithms, like you did copy. You can even mix it, like a variant of various type erased sub-algorithms for the various kinds of type stored.
This does not support any algorithm on any type polymorphism; one of the two must be enumerated for things to resolve at compile time and not have a runtime check.
Function1 can be called with any type T which will be converted to (void*) to be able to add to the list but with this I lose the original pointer type (I need t store tham in one linkedlist because I cannot create one for every possible type). So somehow I need to save the type of the pointer as well. I know that it cant be done using c++. Can anyone suggest an alternative solution?
class MyClass
{
template<class T>
void function1(T* arg1)
{
myList.add((void*)arg);
}
void function2()
{
for(int i = 0; i < myList.size(); i++)
{
myList.get(i);
//restore the original pointer type
}
}
STLinkedlist<void*> myList;
}
The usual way to handle these kinds of problems is by using a public interface, in C++ this is done through inheritance. This can be a drag, especially in constrained situations, where a full class/interface hierarchy would provide too much code/runtime overhead.
In comes Boost.Variant, which allows you to use the same object to store different types. If you need even more freedom, use Boost.Any. For a comparison, see e.g. here.
At the end of the day (or better: rather sooner than later), I'd try to do things differently so you don't have this problem. It may well be worth it in the end.
If you lost the type info by going void* it is just gone. You can not just restore it.
So you either must store extra information along with the pointer, then use branching code to cast it back, or rather drive design to avoid the loss.
Your case is pretty suspicious that you do not what you really want.
A more usual case is that you want a polymorphic collection. That doesn't store any kind of pointers but those belonging to the same hierarchy. Collection has Base* pointers, and you can use the objects through Base's interface, all calling the proper virtual function without programmer's interaction. And if you really need to cast to the original type you can do it via dynamic_cast safely. Or add some type info support in the base interface.
Function1 can be called with any type T which will be converted to (void*) to be able to add to the list but with this I lose the original pointer type (I need t store tham in one linkedlist because I cannot create one for every possible type).
You're having the XY problem. The solution is not to decay your pointers to void* and store type information.
You simply can create a type for every possible type - you create a template type. You need to define an abstract interface for your "type for every object", then define a template class implementing this interface, that is particularized by type. Finally, you create your custom-type instance on your type of pointer received and store them by base class pointer (where the base class is your interface definition).
All that said, you (normally) shouldn't need to implement this at all, because the functionality is already implemented in boost::any or boost::variant (you will have to choose one of them).
General
Take into consideration, that if you want to store different objects inside a std::vector<void *>, mostly likely your application has a bad design. In this case, I'd think, whether it is really necessary to do it (or how can it be done in another way), rather than searching for the solution, how to do it.
However, there are no fully evil things in C++ (nor in any other language), so if you are absolutely certain, that this is the only solution, here are three possible ways to solve your problem.
Option 1
If you store only pointers to simple types, store the original type along with the pointer by an enum value or simply a string.
enum DataType
{
intType,
floatType,
doubleType
};
std::vector<std::pair<void *, DataType>> myData;
Option 2
If you store mixed data (classes and simple types), wrap your data in some kind of class.
class BaseData
{
public:
virtual ~BaseData() { }
};
class IntData : public BaseData
{
public:
int myData;
};
std::vector<BaseData *> myData;
Later, you'll be able to check the type of your data using dynamic_cast.
Option 3
If you store only classes, store them simply as a pointer to their base class and dynamic_cast your way out.
You could use boost::any to store any type in your list instead of use void*. It's not exactly what you want but I don't think you can restore the type in run time (as Kerrek said, it's not Java).
class MyClass
{
template<class T>
void function1(T arg1)
{
myList.add(arg);
}
template<class T>
T get(int i)
{
return boost::any_cast<T>(myList.get(i));
}
STLinkedlist<boost::any> myList;
};
What, if any, c++ constructs are there for listing the ancestors of a class at runtime?
Basically, I have a class which stores a pointer to any object, including possibly a primitive type (somewhat like boost::any, which I don't want to use because I need to retain ownership of my objects). Internally, this pointer is a void*, but the goal of this class is to wrap the void* with runtime type-safety. The assignment operator is templated, so at assignment time I take the typeid() of the incoming pointer and store it. Then when I cast back later, I can check the typeid() of the cast type against the stored type_info. If it mismatches, the cast will throw an exception.
But there's a problem: It seems I lose polymorphism. Let's say B is a base of D. If I store a pointer to D in my class, then the stored type_info will also be of D. Then later on, I might want to retrieve a B pointer. If I use my class's method to cast to B*, then typeid(B) == typeid(D) fails, and the cast raises an exception, even though D->B conversion is safe. Dynamic_cast<>() doesn't apply here, since I'm operating on a void* and not an ancestor of B or D.
What I would like to be able to do is check is_ancestor(typeid(B), typeid(D)). Is this possible? (And isn't this what dynamic_cast<> is doing behind the scenes?)
If not, then I am thinking of taking a second approach anyway: implement a a class TypeInfo, whose derived classes are templated singletons. I can then store whatever information I like in these classes, and then keep pointers to them in my AnyPointer class. This would allow me to generate/store the ancestor information at compile time in a more accessible way. So failing option #1 (a built-in way of listing ancestors given only information available at runtime), is there a construct/procedure I can use which will allow the ancestor information to be generated and stored automatically at compile-time, preferably without having to explicitly input that "class A derives from B and C; C derives from D" etc.? Once I have this, is there a safe way to actually perform that cast?
I had a similar problem which I solved through exceptions! I wrote an article about that:
Part 1, Part 2 and code
Ok. Following Peter's advise the outline of the idea follows. It relies on the fact that if D derives from B and a pointer to D is thrown, then a catch clause expecting a pointer to B will be activated.
One can then write a class (in my article I've called it any_ptr) whose template constructor accepts a T* and stores a copy of it as a void*. The class implements a mechanism that statically cast the void* to its original type T* and throws the result. A catch clause expecting U* where U = T or U is a base of T will be activated and this strategy is the key to implementing a test as in the original question.
EDIT: (by Matthieu M. for answers are best self-contained, please refer to Dr Dobbs for the full answer)
class any_ptr {
void* ptr_;
void (*thr_)(void*);
template <typename T>
static void thrower(void* ptr) { throw static_cast<T*>(ptr); }
public:
template <typename T>
any_ptr(T* ptr) : ptr_(ptr), thr_(&thrower<T>) {}
template <typename U>
U* cast() const {
try { thr_(ptr_); }
catch (U* ptr) { return ptr; }
catch (...) {}
return 0;
}
};
The information is (often) there within the implementation. There's no standard C++ way to access it though, it's not exposed. If you're willing to tie yourself to specific implementations or sets of implementations you can play a dirty game to find the information still.
An example for gcc, using the Itanium ABI is:
#include <cassert>
#include <typeinfo>
#include <cxxabi.h>
#include <iostream>
bool is_ancestor(const std::type_info& a, const std::type_info& b);
namespace {
bool walk_tree(const __cxxabiv1::__si_class_type_info *si, const std::type_info& a) {
return si->__base_type == &a ? true : is_ancestor(a, *si->__base_type);
}
bool walk_tree(const __cxxabiv1::__vmi_class_type_info *mi, const std::type_info& a) {
for (unsigned int i = 0; i < mi->__base_count; ++i) {
if (is_ancestor(a, *mi->__base_info[i].__base_type))
return true;
}
return false;
}
}
bool is_ancestor(const std::type_info& a, const std::type_info& b) {
if (a==b)
return true;
const __cxxabiv1::__si_class_type_info *si = dynamic_cast<const __cxxabiv1::__si_class_type_info*>(&b);
if (si)
return walk_tree(si, a);
const __cxxabiv1::__vmi_class_type_info *mi = dynamic_cast<const __cxxabiv1::__vmi_class_type_info*>(&b);
if (mi)
return walk_tree(mi, a);
return false;
}
struct foo {};
struct bar : foo {};
struct baz {};
struct crazy : virtual foo, virtual bar, virtual baz {};
int main() {
std::cout << is_ancestor(typeid(foo), typeid(bar)) << "\n";
std::cout << is_ancestor(typeid(foo), typeid(baz)) << "\n";
std::cout << is_ancestor(typeid(foo), typeid(int)) << "\n";
std::cout << is_ancestor(typeid(foo), typeid(crazy)) << "\n";
}
Where I cast the type_info to the real type that's used internally and then recursively used that to walk the inheritance tree.
I wouldn't recommend doing this in real code, but as an exercise in implementation details it's not impossible.
First, what you are asking for cannot be implemented just on top of type_info.
In C++, for a cast to occur from one object to another, you need more than blindly assuming a type can be used as another, you also need to adjust the pointer, because of multi-inheritance (compile-time offset) and virtual inheritance (runtime offset).
The only way to safely cast a value from a type into another, is to use static_cast (works for single or multi-inheritance) and dynamic_cast (also works for virtual inheritance and actually checks the runtime values).
Unfortunately, this is actually incompatible with type erasure (the old template-virtual incompatibility).
If you limit yourself to non-virtual inheritance, I think it should be possible to achieve this by storing the offsets of conversions to various bases in some Configuration data (the singletons you are talking about).
For virtual inheritance, I can only think of a map of pairs of type_info to a void* (*caster)(void*).
And all this requires enumerating the possible casts manually :(
It is not possible using std::type_info since it does not provide a way to query inheritance information or to convert a std::type_info object to its corresponding type so that you could do the cast.
If you do have a list of all possible types you need to store in your any objects use boost::variant and its visitor.
While I can't think of any way to implement option #1, option #2 should be feasible if you can generate a compile-time list of the classes you would like to use. Filter this type list with boost::MPL and the is_base_of metafunction to get a list of valid-cast typeids, which can be compared to the saved typeid.