I was curious to know the internals of template class compilation in specific circumstances. I ask this because I'd like to extend some existing classes.
For example, let's assume a starting
template<typename T>
class LargeClass {
private:
std::unique_ptr<T> data;
// many other fields
public:
const std::unique_ptr<T>& getData() { return data; }
void setData(T* value) { data.reset(value); }
// many other methods that don't depend on T
}
This example makes me think that, since sizeof(std::unique_ptr<T>) == sizeof(T*) (without a custom deleter) then sizeof(LargeClass<T1>) == sizeof(LargeClass<T2>) for any T1 and T2. Which implies that all offsetof(field, LargeClass<T>) are the same for any field and T.
So why the compiler would create a copy of each LargeClass method if they don't depend on T?
A different approach, like
class LargeClass {
private:
T data
...
private
...
}
instead should force the compiler to create multiple definitions of the methods for different types since sizeof(T) could change, but using an std::aligned_storage with a static_assert (to avoid storing too large data) could make this fallback to the first case. Would a compiler be smart enough to realise it?
In general I was wondering if a compiler is smart enough to avoid generating multiple definition of a template class method if the type variable is not used and the class structure doesn't change (or at least doesn't change for accessing the fields accessed in the method) or not. Does the standard enforce anything about it?
Related
I have a custom container class that is templated:
template<typename T>
class MyContainer {
T Get();
void Put(T data);
};
I would like to pass a pointer to this container to a function that will access the container's data as generic data - i.e. char* or void*. Think serialization. This function is somewhat complicated so it would be nice to not specify it in the header due to the templates.
// Errors of course, no template argument
void DoSomething(MyContainer *container);
I'm ok with requiring users to provide a lambda or subclass or something that performs the conversion. But I can't seem to come up with a clean way of doing this.
I considered avoiding templates altogether by making MyContainer hold a container of some abstract MyData class that has a virtual void Serialize(void *dest) = 0; function. Users would subclass MyData to provide their types and serialization but that seems like it's getting pretty complicated. Also inefficient since it requires storing pointers to MyData to avoid object slicing and MyData is typically pretty small and the container will hold large amounts (a lot of pointer storage and dereferencing).
You don't need any char* or void* or inheritance.
Consider this simplified implementation:
template <class T>
void Serialize (std::ostream& os, const MyContainer<T>& ct) {
os << ct.Get();
}
Suddenly this works for any T that has a suitable operator<< overload.
What about user types that don't have a suitable operator<< overload? Just tell the users to provide one.
Of course you can use any overloaded function. It doesn't have to be named operator<<. You just need to communicate its name and signature to the users and ask them to overload it.
You can introduce a non-template base class for the container with a pure virtual function that returns a pointer to raw data and implement it in your container:
class IDataHolder
{
public:
virtual ~IDataHolder(); // or you can make destructor protected to forbid deleteing by pointer to base class
virtual const unsigned char* GetData() const = 0;
};
template<typename T>
class MyContainer : public IDataHolder
{
public:
T Get();
void Put(T data);
const unsigned char* GetData() const override { /* cast here internal data to pointer to byte */}
};
void Serialize(IDataHolder& container)
{
const auto* data = container.GetData();
// do the serialization
}
I would like to pass a pointer to this container to a function that will access the container's data as generic data - i.e. char* or void*. Think serialization.
Can't be done in general, because you don't know anything about T. In general, types cannot be handled (e.g. copied, accessed, etc.) as raw blobs through a char * or similar.
Therefore, you would need to restrict what T can be, ideally enforcing it, otherwise never using it for Ts that would trigger undefined behavior. For instance, you may want to assert that std::is_trivially_copyable_v<T> holds. Still, you will have to consider other possible issues when handling data like that, like endianness and packing.
This function is somewhat complicated so it would be nice to not specify it in the header due to the templates.
Not sure what you mean by this. Compilers can handle very easily headers, and in particular huge amounts of template code. As long as you don't reach the levels of e.g. some Boost libraries, your compile times won't explode.
I considered avoiding templates altogether by making MyContainer hold a container of some abstract MyData class that has a virtual void Serialize(void *dest) = 0; function. Users would subclass MyData to provide their types and serialization but that seems like it's getting pretty complicated. Also inefficient since it requires storing pointers to MyData to avoid object slicing and MyData is typically pretty small and the container will hold large amounts (a lot of pointer storage and dereferencing).
In general, if you want a template, do a template. Using dynamic dispatching for this will probably kill performance, specially if you have to go through dispatches for even simple types.
As a final point, I would suggest taking a look at some available serialization libraries to see how they achieved it, not just in terms of performance, but also in terms of easy of use, integration with existing code, etc. For instance, Boost Serialization and Google Protocol Buffers.
I have a limited set of very different types, from which I want to store instances in a single collection, specifically a map. To this end, I use the type erasure idiom, ie. I have a non-templated base class from which the templated, type specific class inherits:
struct concept
{
virtual std::unique_ptr<concept> copy() = 0; // example member function
};
template <typename T>
struct model : concept
{
T value;
std::unique_ptr<concept> copy() override { ... }
}
I then store unique_ptrs to concept in my map. To retrieve the value, I have a templated function which does a dynamic cast to the specified type.
template <typename T>
void get(concept& c, T& out) {
auto model = dynamic_cast<model<T>>(&c);
if (model == nullptr) throw "error, wrong type";
out = model->value;
}
What I don't like about this solution is, that specifying a wrong T is only detected at runtime. I'd really really like this to be done at compile time.
My options are as I see the following, but I don't think they can help here:
Using ad hoc polymorphism by specifying free functions with each type as an overload, or a template function, but I do not know where to store the result.
Using CRTP won't work, because then the base class would need to be templated.
Conceptually I would need a virtual function which takes an instance of a class where the result will be stored. However since my types are fundamentally different, this class would need to be templated, which does not work with virtual.
Anyways, I'm not even sure if this is logically possible, but I would be very glad if there was a way to do this.
For a limited set of types, your best option is variant. You can operate on a variant most easily by specifying what action you would take for every single variant, and then it can operate on a variant correctly. Something along these lines:
std::unordered_map<std::string, std::variant<Foo, Bar>> m;
m["a_foo"] = Foo{};
m["a_bar"] = Bar{};
for (auto& e : m) {
std::visit(overloaded([] (Foo&) { std::cerr << "a foo\n"; }
[] (Bar&) { std::cerr << "a bar\n"; },
e.second);
}
std::variant is c++17 but is often available in the experimental namespace beforehand, you can also use the version from boost. See here for the definition of overloaded: http://en.cppreference.com/w/cpp/utility/variant/visit (just a small utility the standard library unfortunately doesn't provide).
Of course, if you are expecting that a certain key maps to a particular type, and want to throw an error if it doesn't, well, there is no way to handle that at compile time still. But this does let you write visitors that do the thing you want for each type in the variant, similar to a virtual in a sense but without needing to actually have a common interface or base class.
You cannot do compile-time type checking for an erased type. That goes against the whole point of type erasure in the first place.
However, you can get an equivalent level of safety by providing an invariant guarantee that the erased type will match the expected type.
Obviously, wether that's feasible or not depends on your design at a higher level.
Here's an example:
class concept {
public:
virtual ~concept() {}
};
template<typename T>
struct model : public concept {
T value;
};
class Holder {
public:
template<typename T>
void addModel() {
map.emplace(std::type_index(typeid(T)), std::make_unique<model<T><());
}
template<typename T>
T getValue() {
auto found = types.find(std::type_index(typeid(T)));
if(found == types.end()) {
throw std::runtime_error("type not found");
}
// no need to dynamic cast here. The invariant is covering us.
return static_cast<model<T>*>(found->second.get())->value;
}
private:
// invariant: map[type] is always a model<type>
std::map<std::type_index, std::unique_ptr<concept>> types;
};
The strong encapsulation here provides a level of safety almost equivalent to a compile-time check, since map insertions are aggressively protected to maintain the invariant.
Again, this might not work with your design, but it's a way of handling that situation.
Your runtime check occurs at the point where you exit type erasure.
If you want to compile time check the operation, move it within the type erased boundaries, or export enough information to type erase later.
So enumerate the types, like std variant. Or enumerate the algorithms, like you did copy. You can even mix it, like a variant of various type erased sub-algorithms for the various kinds of type stored.
This does not support any algorithm on any type polymorphism; one of the two must be enumerated for things to resolve at compile time and not have a runtime check.
I am struggling with allowing user to select data type template will be created as.
Since template type must be defined on compile, I must specify data type template will use eg(string,int, so on), but that means I cannot change it latter on, from lets say string to int even if my template supports it, because template class object was declared as string.
My class declaration below:
template <class T>
class MyHashTable
{
public:
string deleted="deleted";
unsigned short tableSize;
// array of vectors, hash table container
vector<T>* myTable;
vector<T>* deletionTable;
MyHashTable(unsigned short tableSize) : myTable(new vector<T>[tableSize]), deletionTable(new vector<T>[tableSize])
{
this->tableSize=tableSize;
}
object declaration outside class
MyHashTable <string>* myChainedTable=NULL ;
string tableType;
object initialization
if (myChainedTable)
{
delete myChainedTable;
myChainedTable=NULL;
}
getType();
if (!myChainedTable)
{
if (tableType=="string")
myChainedTable= new MyHashTable<string>(length);
if (tableType=="char")
MyHashTable<char> myChainedTable(length); // no difference with or without using new keyword
if (tableType=="double")
MyHashTable<double> myChainedTable(length);
if (tableType=="float")
MyHashTable<float> myChainedTable(length);
if (tableType=="int")
MyHashTable<int> myChainedTable(length);
cout<<tableType<<" table of size "<< length<<" created"<<endl;
I attempted passing class object to functions instead of having it as global variable, but couldnt get it work either.
What I really need is single template object that can have: int,string,char,double,float types, I have 3 functions that need to have access to template class object, and having 5 different objects and 200 lines of if statements for each situation sounds like worst possible solution.
I been stuck on this for a while and just cant figure out how to do it and any help will be appreciated.
void getType()
{
cout<<"Enter table type, types available: int, char, float, double, string.\n";
tableType=getInput();
while((tableType != "int")&&(tableType !="float")&&(tableType !="double")&&(tableType!="char")&&(tableType !="string"))
{
cout<<"Invalid type, please try again "<<endl;;
tableType=getInput();
}
}
Your question is at the boarder between templates and variants.
The template is compile time. So you have to choose at compile time the type you want for your object. Your conditional approach can't work (see comments to question).
On the other side, you seem to need a dynamic choice of type at runtime.
If you want to go on on template way: (edit based on comments)
You'd need to have all the templates inherit from a single polymorphic base class (one common interface with virtual functions). Example:
class MyHashBase // common base class for all templates
{
public:
virtual void addElement(void *ptrelem) = 0; // adding an element must be implemented by template. With void* since future template type unknown from base class
virtual void displayAll() = 0;
};
The templates would need then implement the virtual functions:
template <class T>
class MyHashTable : public MyHashBase
{
public:
unsigned short tableSize;
vector<T>* myTable; // I leave it as it is, but you could implement these as vector<T> instead of vector<T>*
vector<T>* deletionTable;
MyHashTable(unsigned short tableSize) : myTable(new vector<T>[tableSize]), deletionTable(new vector<T>[tableSize]), tableSize(tableSize)
{ }
void addElement(void* ptrelem)
{ myTable->push_back(*reinterpret_cast<T*>(ptrelem)); } // reinterpret the void* of the common interface as a T*
void displayAll()
{ copy(myTable->begin(), myTable->end(), ostream_iterator<T>(cout, "\n")); }
};
You could then have your myChainedTable be a pointer to the common base type, and intialise this pointer in the way you did with the string case (i.e. using new).
MyHashBase *myChainedTable = nullptr;
//...
if (tableType == "string")
myChainedTable = new MyHashTable<string>(length);
else if (tableType == "double")
myChainedTable = new MyHashTable<double>(length);
//...
You could then use the common API, for example if tableType is "double":
double d1 = 3.1415, d2 = 1.4142;
myChainedTable->addElement(&d1); // ATTENTION: you must ensure to provide pointer to the correct data type
myChainedTable->addElement(&d2);
myChainedTable->displayAll();
You'll certainly have a coupe of if required in the calling code, but you could reduce them to minimum by carefully designing the base class (for example, you could add a virtual clone function, to duplicate the data without need to know the type by the caller).
However, using a single signature for the common functions of the base class is cumbersome. To make the virtualisation possible you need to pass parameters through void* pointer which is not so nice and rather error prone.
Alternate way with variants
You could also use boost variants which are meant for managing objects with dynamic definition of types.
In this case you would not need template for your own data structure. You would create a MyHashTable with elements of type boost::variant< int, std::string, ... >.
You could then access to the right value of the object if you know its type (as in your myChainedTable) by using: boost::get<int> (element) (or boost::get<string>(), ...).
If you don't know the type on an element you could use the concept of "visitor" to chose automatically the appropriate function to exectue depending on the type.
Edit: alternate way with unions:
If you're not allowed to use variants another alternative could be use a union. I don't know the topic of you rassignment, but you have the choice whether you use a union to define the elements (like the variants, without templates) or to use a template type as you did, but define myChainedTable to be a union of pointers to the different template instantiations. But yes, it requires a lot of ifs...
Templates are resolved at compile time. Your container type is resolved at runtime. Templates are clearly not the solution here. The first thing that comes to my mind is a combination of boost::any and std::vector instead.
I hope the headline isn't too confusing. What I have is a class StorageManager containing a list of objects of classes derived from Storage. Here is an example.
struct Storage {}; // abstract
class StorageManager
{
private:
map<string, unique_ptr<Storage>> List; // store all types of storage
public:
template <typename T>
void Add(string Name) // add new storage with name
{
List.insert(make_pair(Name, unique_ptr<Storage>(new T())));
}
Storage* Get(string Name) // get storage by name
{
return List[Name].get();
}
};
Say Position is a special storage type.
struct Position : public Storage
{
int X;
int Y;
};
Thanks to the great answers on my last question the Add function already works. What I want to improve is the Get function. It reasonable returns a pointer Storage* what I can use like the following.
int main()
{
StorageManager Manager;
Manager.Add<Position>("pos"); // add a new storage of type position
auto Strge = Manager.Get("pos"); // get pointer to base class storage
auto Pstn = (Position*)Strge; // convert pointer to derived class position
Pstn->X = 5;
Pstn->Y = 42;
}
It there a way to get rid of this pointer casting by automatically returning a pointer to the derived class? Maybe using templates?
use:
template< class T >
T* Get(std::string const& name)
{
auto i = List.find(name);
return i == List.end() ? nullptr : static_cast<T*>(i->second.get());
}
And then in your code:
Position* p = Manager.Get<Position>("pos");
I don't see what you can do for your Get member function besides what #BigBoss already pointed out, but you can improve your Add member to return the used storage.
template <typename T>
T* Add(string Name) // add new storage with name
{
T* t = new T();
List.insert(make_pair(Name, unique_ptr<Storage>(t)));
return t;
}
// create the pointer directly in a unique_ptr
template <typename T>
T* Add(string Name) // add new storage with name
{
std::unique_ptr<T> x{new T{}};
T* t = x.get();
List.insert(make_pair(Name, std::move(x)));
return t;
}
EDIT The temporary prevents us from having to dynamic_cast.
EDIT2 Implement MatthieuM's suggestion.
You can also further improve the function by accepting a value of the
type to be inserted, with a default argument, but that might incur an
additional copy.
When you have a pointer or reference to an object of some class, all you know is that the actual runtime object it references is either of that class or of some derived class. auto cannot know the runtime type of an object at compile time, because the piece of code containing the auto variable could be in a function that is run twice -- once handling an object of one runtime type, another handling an object with a different runtime type! The type system can't tell you what exact types are in play in a language with polymorphism -- it can only provide some constraints.
If you know that the runtime type of an object is some particular derived class (as in your example), you can (and must) use a cast. (It's considered preferable to use a cast of the form static_cast<Position*>, since casts are dangerous, and this makes it easier to search for casts in your code.)
But generally speaking, doing this a lot is a sign of poor design. The purpose of declaring a base class and deriving other class types from it is to enable objects of all of these those types to be treated the same way, without casting to a particular type.
If you want to always have the correct derived type at compile time without ever using casts, you have no choice but to use a separate collection of that type. In this case, there is probably no point deriving Position from Storage.
If you can rearrange things so that everything that a caller of StorageManager::Get() needs to do with a Position can be done by calling functions that don't specify Position-specific information (such as co-ordinates), you can make these functions into virtual functions in Storage, and implement Position-specific versions of them in Position. For example, you could make a function Storage::Dump() which writes its object to stdout. Position::Dump() would output X and Y, while the implementations of Dump() for other conceivable derived classes would output different information.
Sometimes you need to be able to work with an object that could be one of several essentially unrelated types. I suspect that may be the case here. In that case, boost::variant<> is a good way to go. This library provides a powerful mechanism called the Visitor pattern, which allows you to specify what action should be taken for each of the types that a variant object could possibly be.
Apart from the fact that this looks like a terrible idea... let's see what we can do to improve the situation.
=> It's a bad idea to require default construction
template <typename T>
T& add(std::string const& name, std::unique_ptr<T> element) {
T& t = *element;
auto result = map.insert(std::make_pair(name, std::move(element)));
if (result.second == false) {
// FIXME: somehow add the name here, for easier diagnosis
throw std::runtime_error("Duplicate element");
}
return t;
}
=> It's a bad idea to downcast blindly
template <typename T>
T* get(std::string const& name) const {
auto it = map.find(name);
return it != map.end() ? dynamic_cast<T*>(it->second.get()) : nullptr;
}
But frankly, this system is quite full of holes. And probably unnecessary in the first place. I encourage you to review the general problem an come up with a much better design.
I am implementing a task runtime system that maintains buffers for user-provided objects of various types. In addition, all objects are wrapped before they are stored into the buffers. Since the runtime doesn't know the types of objects that the user will provide, the Wrapper and the Buffer classes are templated:
template <typename T>
class Wrapper {
private:
T mdata;
public:
Wrapper() = default;
Wrapper(T& user_data) : mdata(user_data) {}
T& GetData() { return mdata; }
...
};
template <typename T>
class Buffer {
private:
std::deque<Wrapper<T>> items;
public:
void Write(Wrapper<T> wd) {
items.push_back(wd);
}
Wrapper<T> Read() {
Wrapper<T> tmp = items.front();
items.pop_front();
return tmp;
}
...
};
Now, the runtime system handles the tasks, each of which operates on a subset of aforementioned buffers. Thus, each buffer is operated by one or more tasks. This means that a task must keep references to the buffers since the tasks may share buffers.
This is where my problem is:
1) each task needs to keep references to a number of buffers (this number is unknown in compile time)
2) the buffers are of different types (based on the templeted Buffer class).
3) the task needs to use these references to access buffers.
There is no point to have a base class to the Buffer class and then use base class pointers since the methods Write and Read from the Buffer class are templeted and thus cannot be virtual.
So I was thinking to keep references as void pointers, where the Task class would look something like:
class Task {
private:
vector<void *> buffers;
public:
template<typename T>
void AddBuffer(Buffet<T>* bptr) {
buffers.push_back((void *) bptr);
}
template<typename T>
Buffer<T>* GetBufferPtr(int index) {
return some_way_of_cast(buffers[index]);
}
...
};
The problem with this is that I don't know how to get the valid pointer from the void pointer in order to access the Buffer. Namely, I don't know how to retain the type of the object pointed by buffers[index].
Can you help me with this, or suggest some other solution?
EDIT: The buffers are only the implementation detail of the runtime system and the user is not aware of their existence.
In my experience, when the user types are kept in user code, run-time systems handling buffers do not need to worry about the actual type of these buffer. Users can invoke operations on typed buffers.
class Task {
private:
vector<void *> buffers;
public:
void AddBuffer(char* bptr) {
buffers.push_back((void *) bptr);
}
char *GetBufferPtr(int index) {
return some_way_of_cast(buffers[index]);
}
...
};
class RTTask: public Task {
/* ... */
void do_stuff() {
Buffer<UserType1> b1; b1Id = b1.id();
Buffer<UserType2> b2; b2Id = b2.id();
AddBuffer(cast(&b1));
AddBuffer(cast(&b2));
}
void do_stuff2() {
Buffer<UserType1> *b1 = cast(GetBufferPtr(b1Id));
b1->push(new UserType1());
}
};
In these cases casts are in the user code. But perhaps you have a different problem. Also the Wrapper class may not be necessary if you can switch to pointers.
What you need is something called type erasure. It's way to hide the type(s) in a template.
The basic technique is the following:
- Have an abstract class with the behavior you want in declared in a type independent maner.
- Derive your template class from that class, implement its virtual methods.
Good news, you probably don't need to write your own, there boost::any already. Since all you need is get a pointer and get the object back, that should be enough.
Now, working with void* is a bad idea. As perreal mentioned, the code dealing with the buffers should not care about the type though. The good thing to do is to work with char*. That is the type that is commonly used for buffers (e.g. socket apis). It is safer than too: there is a special rule in the standard that allows safer conversion to char* (see aliasing rules).
This isn't exactly an answer to your question, but I just wanted to point out that the way you wrote
Wrapper<T> Read() {
makes it a mutator member function which returns by value, and as such, is not good practice as it forces the user write exception unsafe code.
For the same reason the STL stack::pop() member function returns void, not the object that was popped off the stack.