Is there any way in C++ to create runtime-modificable collection that matches integer to template type?
First I though about:
std::map<uint32_t, std::type_index> type;
but std::type_index has very small usefulness. I need to be able to cast by that type and use it as argument for std::is_convertible
So the api should be able to something like that:
my_collection.add<1234, std::string>();
my_collection.get<1234>()::type // type is std::string
Or actually anything else close enough I can use in runtime :)
No.
If your list of types you support can be finitely and centrally enumerated, std::variant or boost::variant or a roll-your-own can be used.
If your list of operations on the type you support can be finitely and centrally enumerated, std::any or boost::any together with type erasure techniques can be used to solve your problem. Here is an overly generic system to type erase nearly any single-dispatch operation.
You can use std::any or boots::any to have a non-centrally yet locally finitely enumerated list of types, but only the exact types known at each enumeration location are in play at that location.
You can do a combination of the above.
Failing that, you are entering the world of runtime compiling of dynamic C++ libraries and passing the result, and/orraw C++ with source code, around. Doing on the fly compiling.
All of this will depend on what your exact, actual problem is for which that you came up with this as a "solution"; there is probably going to be a related question for which your answer is "yes", and that related question is probably what you actually need to solve.
But the answer to your question as posed is simply "No".
Related
I try to create a c++ flex/bison parser. I used this tutorial as a starting point and did not change any bison/flex configurations. I am stuck now to the point of trying to unit test the lexer.
I have a function in my unit tests that directly calls yylex, and checks the result of it:
private: static void checkIntToken(MyScanner &scanner, Compiler *comp, unsigned long expected, unsigned char size, char isUnsigned, unsigned int line, const std::string &label) {
yy::MyParser::location_type loc;
yy::MyParser::semantic_type semantic; // <---- is seems like the destructor of this variable causes the crash
int type = scanner.yylex(&semantic, &loc, comp);
Assert::equals(yy::MyParser::token::INT, type, label + "__1");
MyIntToken* token = semantic.as<MyIntToken*>();
Assert::equals(expected, token->value, label + "__2");
Assert::equals(size, token->size, label + "__3");
Assert::equals(isUnsigned, token->isUnsigned, label + "__4");
Assert::equals(line, loc.begin.line, label + "__5");
//execution comes to this point, and then, program crashes
}
The error message is:
program: ../src/__autoGenerated__/MyParser.tab.hh:190: yy::variant<32>::~variant() [S = 32]: Assertion `!yytypeid_' failed.
I have tried to follow the logic in the auto-generated bison files, and make some sense out of it. But I did not succeed on that and ultimately gave up. I searched then for any advice on the web about this error message but did not find any.
The location indicated by the error has the following code:
~variant (){
YYASSERT (!yytypeid_);
}
EDIT: The problem disappears only if I remove the
%define parse.assert
option from the bison file. But I am not sure if this is a good idea...
What is the proper way to obtain the value of the token generated by flex, for unit testing purposes?
Note: I've tried to explain bison variant types to the best of my knowledge. I hope it is accurate but I haven't used them aside from some toy experiments. It would be an error to assume that this explanation in any way implies an endorsement of the interface.
The so-called "variant" type provided by bison's C++ interface is not a general-purpose variant type. That was a deliberate decision based on the fact that the parser is always able to figure out the semantic type associated with a semantic value on the parser stack. (This fact also allows a C union to be used safely within the parser.) Recording type information within the "variant" would therefore be redundant. So they don't. In that sense, it is not really a discriminated union, despite what one might expect of a type named "variant".
(The bison variant type is a template with an integer (non-type) template argument. That argument is the size in bytes of the largest type which is allowed in the variant; it does not in any other way specify the possible types. The semantic_type alias serves to ensure that the same template argument is used for every bison variant object in the parser code.)
Because it is not a discriminated union, its destructor cannot destruct the current value; it has no way to know how to do that.
This design decision is actually mentioned in the (lamentably insufficient) documentation for the Bison "variant" type. (When reading this, remember that it was originally written before std::variant existed. These days, it would be std::variant which was being rejected as "redundant", although it is also possible that the existence of std::variant might have had the happy result of revisiting this design decision). In the chapter on C++ Variant Types, we read:
Warning: We do not use Boost.Variant, for two reasons. First, it appeared unacceptable to require Boost on the user’s machine (i.e., the machine on which the generated parser will be compiled, not the machine on which bison was run). Second, for each possible semantic value, Boost.Variant not only stores the value, but also a tag specifying its type. But the parser already “knows” the type of the semantic value, so that would be duplicating the information.
Therefore we developed light-weight variants whose type tag is external (so they are really like unions for C++ actually).
And indeed they are. So any use of a bison "variant" must have a definite type:
You can build a variant with an argument of the type to build. (This is the only case where you don't need a template parameter, because the type is deduced from the argument. You would have to use an explicit template parameter only if the argument were not of the precise type; for example, an integer of lesser rank.)
You can get a reference to the value of known type T with as<T>. (This is undefined behaviour if the value has a different type.)
You can destruct the value of known type T with destroy<T>.
You can copy or move the value from another variant of known type T with copy<T> or move<T>. (move<T> involves constructing and then destructing a T(), so you might not want to do it if T had an expensive default constructor. On the whole, I'm not convinced by the semantics of the move method. And its name conflicts semantically with std::move, but again it came first.)
You can swap the values of two variants which both have the same known type T with swap<T>.
Now, the generated parser understands all these restrictions, and it always knows the real types of the "variants" it has at its disposal. But you might come along and try to do something with one of these objects in a way that violates a constraint. Since the object really doesn't have any way to check the constraint, you'll end up with undefined behaviour which will probably have some disastrous eventual consequence.
So they also implemented an option which allows the "variant" to check the constraints. Unsurprisingly, this consists of adding a discriminator. But since the discriminator is only used to validate and not to modify behaviour, it is not a small integer which chooses between a small number of known alternatives, but rather a pointer to a std::typeid (or NULL if the variant does not yet contain a value.) (To be fair, in most cases alignment constraints mean that using a pointer for this purpose is no more expensive than using a small enum. All the same...)
So that's what you're running into. You enabled assertions with %define parse.assert; that option was provided specifically to prevent you from doing what you are trying to do, which is let the variant object's destructor run before the variant's value is explicitly destructed.
So the "correct" way to avoid the problem is to insert an explicit call at the end of the scope:
// execution comes to this point, and then, without the following
// call, the program will fail on an assertion
semantic.destroy<MyIntType*>();
}
With the parse assertion enabled, the variant object will be able to verify that the types specified as template parameters to semantic.as<T> and semantic.destroy<T> are the same types as the value stored in the object. (Without parse.assert, that too is your responsibility.)
Warning: opinion follows.
In case anyone reading this cares, my preference for using real std::variant types comes from the fact that it is actually quite common for the semantic value of an AST node to require a discriminated union. The usual solution (in C++) is to construct a type hierarchy which is, in some ways, entirely artificial, and it is quite possible that std::variant can better express the semantics.
In practice, I use the C interface and my own discriminated union implementation.
Ive spent the day reading notes and watching a video on boost::fusion and I really don't get some aspects to it.
Take for example, the boost::fusion::has_key<S> function. What is the purpose of having this in boost::fusion? Is the idea that we just try and move as much programming as possible to happen at compile-time? So pretty much any boost::fusion function is the same as the run-time version, except it now evaluates at compile time? (and we assume doing more at compile-time is good?).
Related to boost::fusion, i'm also a bit confused why metafunctions always return types. Why is this?
Another way to look at boost::fusion is to think of it as "poor man introspection" library. The original motivation for boost::fusion comes from the direction of boost::spirit parser/generator framework, in particular the need to support what is called "parser attributes".
Imagine, you've got a CSV string to parse:
aaaa, 1.1
The type, this string parses into, can be described as "tuple of string and double". We can define such tuples in "plain" C++, either with old school structs (struct { string a; double b; } or newer tuple<string, double>). The only thing we miss is some sort of adapter, which will allow to pass tuples (and some other types) of arbitrary composition to a unified parser interface and expect it to make sense of it without passing any out of band information (such as string parsing templates used by scanf).
That's where boost::fusion comes into play. The most straightforward way to construct a "fusion sequence" is to adapt a normal struct:
struct a {
string s;
double d;
};
BOOST_FUSION_ADAPT_STRUCT(a, (string, s)(double, d))
The "ADAPT_STRUCT" macro adds the necessary information for parser framework (in this example) to be able to "iterate" over members of struct a to the tune of the following questions:
I just parsed a string. Can I assign it to first member of struct a?
I just parsed a double. Can I assign it to second member of struct a?
Are there any other members in struct a or should I stop parsing?
Obviously, this basic example can be further extended (and boost::fusion supplies the capability) to address much more complex cases:
Variants - let's say parser can encounter either sting or double and wants to assign it to the right member of struct a. BOOST_FUSION_ADAPT_ASSOC_STRUCT comes to the rescue (now our parser can ask questions like "which member of struct a is of type double?").
Transformations - our parser can be designed to accept certain types as parameters but the rest of the programs had changed quite a bit. Yet, fusion metafunctions can be conveniently used to adapt new types to old realities (or vice versa).
The rest of boost::fusion functionality naturally follows from the above basics. fusion really shines when there's a need for conversion (in either direction) of "loose IO data" to strongly typed/structured data C++ programs operate upon (if efficiency is of concern). It is the enabling factor behind spirit::qi and spirit::karma being such an efficient (probably the fastest) I/O frameworks .
Fusion is there as a bridge between compile-time and run-time containers and algorithms. You may or may not want to move some of your processing to compile-time, but if you do want to then Fusion might help. I don't think it has a specific manifesto to move as much as possible to compile-time, although I may be wrong.
Meta-functions return types because template meta-programming wasn't invented on purpose. It was discovered more-or-less by accident that C++ templates can be used as a compile-time programming language. A meta-function is a mapping from template arguments to instantiations of a template. As of C++03 there were are two kinds of template (class- and function-), therefore a meta-function has to "return" either a class or a function. Classes are more useful than functions, since you can put values etc. in their static data members.
C++11 adds another kind of template (for typedefs), but that is kind of irrelevant to meta-programming. More importantly for compile-time programming, C++11 adds constexpr functions. They're properly designed for the purpose and they return values just like normal functions. Of course, their input is not a type, so they can't be mappings from types to something else in the way that templates can. So in that sense they lack the "meta-" part of meta-programming. They're "just" compile-time evaluation of normal C++ functions, not meta-functions.
I am reading about boost type erasure and I am trying to figure out the potential usage. I would like to practice it a bit while I am reading tons of documentations about the topic (it looks a big one). The most quoted area of application that is networking / exchanging data between client and server.
Can you suggest some other example or exercise where I can play I bit with this library?
Type Erasure is useful in an extraordinary amount of situations, to the point where it may actually be thought of as a fundamentally missing language feature that bridges generic and object oriented programming styles.
When we define a class in C++, what we are really defining is both a very specific type and a very specific interface, and that these two things do not necessarily need to be related. A type deals with the data, where as the interface deals with transformations on that data. Generic code, such as in the STL, doesn't care about type, it cares about interface: you can sort anything container or container-like sequence using std::sort, as long as it provides comparison and iterator interface.
Unfortunately, generic code in C++ requires compile time polymorphism: templates. This doesn't help with things which cannot be known until runtime, or things which require a uniform interface.
A simple example is this: how do you store a number of different types in a single container? The simplest mechanism would be to store all of the types in a void*, perhaps with some type information to distinguish them. Another way is to recognize all of these types have the same interface: retrieval. If we could make a single interface for retrieval, then specialize it for each type, then it would be as if part of the type had been erased.
any_iterator is another very useful reason to do this: if you need to iterate over a number of different containers with the same interface, you will need to erase the type of the container out of the type of the iterator. boost::any_range is a subtle enhancement of this, extending it from iterators to ranges, but the basic idea is the same.
In short, any time you need to go from multiple types with a similar interface to a single type with a single interface, you will need some form of type erasure. It is the runtime technique that equates compile time templates.
OK, so I have this tiny little corner of my code where I'd like my function return either of (int, double, CString) to clean up the code a bit.
So I think: No problem to write a little union-like wrapper struct with three members etc. But wait! Haven't I read of boost::variant? Wouldn't this be exactly what I need? This would save me from messing around with a wrapper struct myself! (Note that I already have the boost library available in my project.)
So I fire up my browser, navigate to Chapter 28. Boost.Variant and lo and behold:
The variant class template is a safe, generic, stack-based discriminated union container, offering a simple solution for manipulating an object from a heterogeneous set of types [...]
Great! Exactly what I need!
But then it goes on:
Boost.Variant vs. Boost.Any
Boost.Any makes little use of template metaprogramming techniques (avoiding potentially hard-to-read error messages and significant compile-time processor and memory demands).
[...]
Troubleshooting
"Internal heap limit reached" -- Microsoft Visual C++ -- The compiler option /ZmNNN can increase the memory allocation limit. The NNN is a scaling percentage (i.e., 100 denotes the default limit). (Try /Zm200.)
[...]
Uh oh. So using boost::variant may significantly increase compile-time and generate hard-to-read error messages. What if someone moves my use of boost::variant to a common header, will our project suddenly take lots longer to compile? Am I introducing an (unnecessarily) complex type?
Should I use boost::variant for my simple tiny problem?
Generally, use boost::variant if you do want a discriminated union (any is for unknown types -- think of it as some kind of equivalent to how void* is used in C).
Some advantages include exception handling, potential usage of less space than the sum of the type sizes, type discriminated "visiting". Basically, stuff you'd want to perform on the discriminated union.
However, for boost::variant to be efficient, at least one of the types used must be "easily" constructed (read the documentation for more details on what "easily" means).
Boost.variant is not that complex, IMHO. Yes, it is template based, but it doesn't use any really complex feature of C++. I've used quite a bit and no problem at all. I think in your case it would help better describing what your code is doing.
Another way of thinking is transforming what that function returns into a more semantically rich structure/class that allows interpreting which inner element is interesting, but that depends on your design.
This kind of boost element comes from functional programming, where you have variants around every corner.
It should be a way to have a type-safe approach to returning a kind of value that can be of many precise types. This means that is useful to solve your problem BUT you should consider if it's really what you need to do.
The added value compared to other approaches that tries to solve the same problem should be the type-safety (you won't be able to place whatever you want inside a variant without noticing, in opposition to a void*)
I don't use it because, to me, it's a symptom of bad design.
Either your method should return an object that implements a determinated interface or it should be split in more than one method. Design should be reviewed, anyway.
I'm in the process of creating a class that stores metadata about a particular data source. The metadata is structured in a tree, very similar to how XML is structured. The metadata values can be integer, decimal, or string values.
I'm curious if there is a good way in C++ to store variant data for a situation like this. I'd like for the variant to use standard libraries, so I'm avoiding the COM, Ole, and SQL VARIANT types that are available.
My current solution looks something like this:
enum MetaValueType
{
MetaChar,
MetaString,
MetaShort,
MetaInt,
MetaFloat,
MetaDouble
};
union MetaUnion
{
char cValue;
short sValue;
int iValue;
float fValue;
double dValue;
};
class MetaValue
{
...
private:
MetaValueType ValueType;
std::string StringValue;
MetaUnion VariantValue;
};
The MetaValue class has various Get functions for obtaining the currently stored variant value, but it ends up making every query for a value a big block of if/else if statements to figure out which value I'm looking for.
I've also explored storing the value as only a string, and performing conversions to get different variant types out, but as far as I've seen this leads to a bunch of internal string parsing and error handling which isn't pretty, opens up a big old can of precision and data loss issues with floating point values, and still doesn't eliminate the query if/else if issue stated above.
Has anybody implemented or seen something that's cleaner to use for a C++ variant data type using standard libraries?
As of C++17, there’s std::variant.
If you can’t use that yet, you might want Boost.Variant. A similar, but distinct, type for modelling polymorphism is provided by std::any (and, pre-C++17, Boost.Any).
Just as an additional pointer, you can look for “type erasure”.
While Konrad's answer (using an existing standardized solution) is certainly preferable to writing your own bug-prone version, the boost variant has some overheads, especially in copy construction and memory.
A common customized approach is the following modified Factory Pattern:
Create a Base interface for a generic object that also encapsulates the object type (either as an enum), or using 'typeid' (preferable).
Now implement the interface using a template Derived class.
Create a factory class with a templateized create function with signature:
template <typename _T> Base * Factory::create ();
This internally creates a Derived<_T> object on the heap, and retuns a dynamic cast pointer. Specialize this for each class you want implemented.
Finally, define a Variant wrapper that contains this Base * pointer and defines template get and set functions. Utility functions like getType(), isEmpty(), assignment and equality operators, etc can be appropriately implemented here.
Depending on the utility functions and the factory implementation, supported classes will need to support some basic functions like assignment or copy construction.
You can also go down to a more C-ish solution, which would have a void* the size of a double on your system, plus an enum for which type you're using. It's reasonably clean, but definitely a solution for someone who feels wholly comfortable with the raw bytes of the system.
C++17 now has std::variant which is exactly what you're looking for.
std::variant
The class template std::variant represents a type-safe union. An
instance of std::variant at any given time either holds a value of one
of its alternative types, or in the case of error - no value (this
state is hard to achieve, see valueless_by_exception).
As with unions, if a variant holds a value of some object type T, the
object representation of T is allocated directly within the object
representation of the variant itself. Variant is not allowed to
allocate additional (dynamic) memory.
Although the question had been answered for a long time, for the record I would like to mention that QVariant in the Qt libraries also does this.
Because C++ forbids unions from including types that have non-default
constructors or destructors, most interesting Qt classes cannot be
used in unions. Without QVariant, this would be a problem for
QObject::property() and for database work, etc.
A QVariant object holds a single value of a single type() at a time.
(Some type()s are multi-valued, for example a string list.) You can
find out what type, T, the variant holds, convert it to a different
type using convert(), get its value using one of the toT() functions
(e.g., toSize()) and check whether the type can be converted to a
particular type using canConvert().