I'm trying to process structures which are contained in a file mapping. One of the challenges I'm facing is that some structures have pointers which need a function to get the correct value. It's important that the layout and size of the structure stay the same, and I'm using c++20.
My current solution is as follows.
I have a private data structure, and store the member offsets in a public subclass. The template factory method uses this array of offsets to fix up any pointers.
struct _Data {
uint32_t flags;
uint64_t ptr1;
uint64_t ptr2;
};
struct Data : public {
constexpr static std::size_t PTRS[] = {
offsetof(_Data, ptr1),
offsetof(_Data, ptr2)
};
};
template <class T> T GetStructure(uint64_t addr) {
// Converts the "virtual address" to a pointer in the file mapping
T temp = *(T *)context.translateAddr(addr);
for (auto offset : T::PTRS) {
uint64_t *member = (uint8_t *)temp + offset;
*member = context.getPointer(addr + offset);
}
return temp;
}
I found a similar question here dealing with serialization, which also stores metadata about members in a list.
My main gripe with my solution is how I need to split a definition into two structures. It would be great if I could contain everything within its own definition.
Other solutions I thought were,
Somehow "mark" a member in a way the factory method can detect.
Using a function to return a vector of offsets. This would solve the two definitions problem, but I'm concerned about the performance of constantly calling a function and return a vector.
Using a constexpr function to return a std::array. Faster than returning a vector for sure, but probably slower than a constexpr list like the one shown above.
I can make a separate templated function which returns a a vector of offsets given the structure, but this seems to combine worst of both worlds.
Related
Background
For an embedded project, I want a class that takes a list of structs. This list is known at compile-time, so I shouldn't have to resort to dynamic memory allocation for this.
However, how do I make a struct/class that encapsulates this array without having to use its size as a template parameter?
Templates
My first idea was to do exactly that:
struct Point {
const uint16_t a;
const double b;
};
template<size_t n>
struct Profile {
Array<Point, n> points;
Profile(const Array<Point, n> &points) : points(points) {}
};
Here, Profile is the class that stores/encapsulates the array of points (the 2-member structs). n, the size of the array, is a template parameter.
I'm using this implementation of Array, similar to std::array, btw, because I don't have access to the STL on this embedded platform.
However, no I have another class that uses this Profile that now also has to be templated because Profile is templated with the size of the array:
template<size_t n>
class Runner {
private:
const Profile<n> profile;
public:
Runner(const Profile<n> &profile) : profile(profile) {};
void foo() {
for(auto point : profile.points) {
// do something
}
}
};
As can be seen, this Runner class operates on a Profile and iterates over it. Having to template Runner is not that much of an issue by itself, but this Runner in turn is used by another class in my project, because this other class calls Runner::foo(). Now I have to template that class as well! And classes that use that class, etc.
That's getting out of hand! What started with just one template parameter to specify the size, now propagates through my entire application. Therefore, I don't think this is a good solution.
Question
Is there a way to 'hide' the size of the array in Profile or Runner? Runner only needs to iterate over it, so the size should in principle only affect its implementation, not its public interface. How would I do that, though?
Also, can I avoid having to manually specify n at all, and just pass an array to Profile's constructor and let the compiler figure out how big it is? At compile-time, of course. I feel like this should be possible (given this array is known at compile-time), but I don't know how exactly.
Other approaches
Macros
I could write a macro like
#define n 12
and include that in both the Profile.h and the place where I instantiate a Profile. This feels dirty though, I and would like to avoid macros.
Vector
I could avoid this fuss by just using a std::vector (or equivalent) instead, but that is allocated at run-time on the heap, which I would like to avoid here since it shouldn't be necessary.
Is there a way to 'hide' the size of the array in Profile or Runner?
Yes. The solution is indirection. Instead of storing the object directly, you can point to it. You don't need to know the size of what you're pointing at.
A convenient solution is to point into dynamic storage (for example std::vector) because it allows you to "bind" the lifetime of the dynamically sized object to a member. That's not necessary in general, and you can use automatic storage instead. However, in that case you cannot bind the lifetime of the pointed object, and you must be very careful to not let the pointed object be destroyed before you stop using it.
The indirection can be done at whatever level you prefer. If you do it at the lowest level, you simply store the array outside of Profile. In fact, if all that profile does is contain an array, then you don't need a class for it. Use a generic span:
struct Runner {
span<const Point> profile;
void foo() {
for(auto point : profile) {
// do something
}
}
};
Point points[] {
// ... number of points doesn't matter
};
Runner runner {
.profile = points,
};
By span, I mean something like std::span. If you cannot use the standard library, then use another implementation. It's basically just a pointer and size, with convenient template constructor.
To clarify, you can pick any two, but you cannot have all three of these:
Lifetime of the array bound to the class (safe)
No compiletime constant size
No dynamic storage
1,2 (no 3) = std::vector, RAII
1,3 (no 2) = std::array, templates, no indirection
2,3 (no 1) = std::span, be careful with lifetimes
I'll expand on this comment:
The idea is that Runner takes Profiles no matter their size. Runner needs to iterate over it, but apart from that, its behaviour is always the same. The class using Runner and calling Runner::foo() doesn't need to know the size. The problem with templating Runner is that the class using Runner also needs to be templated, and the classes using that, etc.
This is only a problem when the class is using the templated Runner directly. It has more dependencies than it actually needs. If it doesn't need to know about the size of the array, then it should not know about the size of the array. If runtime polymorphism is an option you can add a base class that allows accessing the array elements, but doesn't need to know anything about the arrays size. The following is only a sketch:
#include <iostream>
struct RunnerInterface {
virtual int* begin() = 0;
virtual int* end() = 0;
virtual ~RunnerInterface(){}
};
template <unsigned size>
struct Runner : RunnerInterface {
int data[size];
int* begin() override { return data; }
int* end() override { return data+size; } // pointer one past the end if fine (it won't get dereferenced)
};
void foo(RunnerInterface& ri) {
for (auto it = ri.begin(); it != ri.end(); ++it){
*it = 42;
}
}
void bar(RunnerInterface& ri){
for (auto it = ri.begin(); it != ri.end(); ++it){
std::cout << *it;
}
}
int main() {
Runner<42> r;
foo(r);
bar(r);
}
Now if a class needs a Runner member, they store a std::unique_ptr<RunnerInterface> and only on construction you need to decide for the size of the array (though you still need to decide for the size somewhere).
I have a class with different functions. This class uses two typedef structs.
typedef struct {
struct head;
struct data1;
} data_struct_1;
typedef struct {
struct head;
struct data2;
} data_struct_2;
If a bool is true the data_struct_2 should be used, otherwise data_struct_1. The boolean is passed, when the object is created and stored as a membervariable in the constructor.
There are now different functions and declarations like
int calc_mean(data_struct_1 ds, int a){ ... }
or
data_struct_1 ds;
ds.data1 = 5;
Is there an elegant way, which allows me to switch between those two structs without implementing dozens of if-cases and duplicate all the relevant functions with the corresponding struct in the header?
First I was thinking about inheritance and overwritting relevant functions if bool is true.. but the header in the functions differs, so this wouldnt work
Maybe there is something like a placeholder or template, but i'm pretty new to c++
If boolean is compile-time value, std::conditional might be used:
template <boolean B>
using my_type = std::conditional_t<B, data_struct_1, data_struct_2>;
but it seems the boolean is runtime value, so I would use std::variant and std::visit:
class C
{
using Data = std::variant<data_struct_1, data_struct_2>;
Data m_data;
public:
C(bool b) : m_data{b ? Data{data_struct_1{}} : Data{data_struct_2{}}} {}
auto mean() const {
return std::visit([](const auto& data){ return calc_mean(data, 42); }, m_data);
}
};
You have a couple of options:
If the types are conceptually the same type of thing (i.e. they have the same interface and could be substituted for one another), you can:
Create an abstract base type and use polymorphism to change which concrete type is instantiated
Change all places that use these types into templates, change which template(s) are instantiated at runtime, and then rely on compile-time "duck-typing" to do the rest
If the types are conceptually different (i.e. they should not have the same interface), you can use a union type (union, std::variant).
Take a look at std::variant.
A variant can hold a number of alternative types, like this:
std::variant<data_struct_1, data_struct_2> m;
The member m can contain either a data_struct_1 or a data_struct_2. The variant will remember which, so there's no need for you to carry that bool around anymore.
The interesting question is now, how do you treat m in your functions? After all, it could be either a data_struct_1 or a data_struct_2 in there. Instead of using branches for distinguishing these cases, variant uses visit: Whenever you encounter a point in the function that depends on the concrete type stored in m, you move that code into two overloaded functions: One function that handles the data_struct_1 case and another that handles the data_struct_2 case. The example on the visit cppreference page shows some possibilities for doing that with varying degrees of syntactic overhead.
The good thing about this approach is that it forces you to always handle both cases properly. You can't just go ahead and write an implementation for one case and then forget about the other, having it blow up later at runtime. It also encourages a style where you clearly separate common parts of your code from type-specific parts, which often results in a cleaner design in the long term.
One possibility is to use std::variant:
typedef std::variant<data_struct_1, data_struct_2> data_struct_1_or_2;
void foo(data_struct_1_or_2 ds) {
if (auto ds1 = std::get_if<data_struct_1>(&ds)) {
// use ds1, which is type data_struct_1*
} else if (auto ds2 = std::get_if<data_struct_2>(&ds)) {
// use d2, which is type data_struct_2*
} else {
throw std::logic_error("Unexpected value in variant");
}
}
void bar(data_struct_1_or_2 ds) {
foo(ds); // Don't need to use an if statement here
}
If you have a data_struct_1 or data_struct_2 and want to pass a reference to the object, rather than a copy, you could use pointers in the std::variant instead (although it'll get a bit confusing with std::get_if because you'll end up with a pointer to a pointer):
typedef std::variant<data_struct_1*, data_struct_2*> data_struct_1_or_2_ptr;
A simple C solution would be a union with a bool tag:
typedef struct {
struct head;
bool type_flag;
union{
struct data1 d1;
struct data2 d2
};
} data_struct;
std::variant is an overkill for this case IMO. Inheritance and runtime polymorphism is a waste of runtime and memory.
Regards,
FM.
My question concerns the possibility of creating a C++ function or similar construct to return a typename (as opposed to an object/reference/pointer of some type, as is typical).
As shown in the code below, I have a generic "Image" class which simply contains a pointer to data (void*), size of the data, and a member variable defining how the data is represented. I need the flexibility to represent 8-bit, 16-bit, ... image types with this same class.
When I want to manipulate image data, I currently have to use a switch/case construct (as shown in the example CopyImage()) function to reinterpret_cast the void* data pointer to the appropriate type prior to making the copy. Unfortunately this same switch/case paradigm keeps getting repeated all over the place as I expand with functions that do things other than CopyImage(). It seems messy and duplicative and limits my ability to add support for more types without replicating additional cases in tons of other functions.
enum class datatype_t {
u8,
u16
};
class Image { // can be an image with 8 or 16-bit pixel representation
public:
void * dataPtr_;
datatype_t dataType_;
size_t pixels_;
};
void CopyImage(Image& source, uint16_t * dst_ptr) { // function to copy image agnostic to data represenation
switch (source.dataType_) {
case datatype_t::u8:
std::copy_n(reinterpret_cast<uint8_t*>(source.dataPtr_), source.pixels_, dst_ptr);
break;
case datatype_t::u16:
std::copy_n(reinterpret_cast<uint16_t*>(source.dataPtr_), source.pixels_, dst_ptr);
break;
}
}
int main()
{
const size_t image_size = 100;
std::vector<Image> image_library;
// create Image objects, populate details, push to image library
Image source_image8;
source_image8.dataType_ = datatype_t::u8;
source_image8.pixels_ = image_size;
source_image8.dataPtr_ = new uint8_t[image_size];
image_library.push_back(std::move(source_image8));
Image source_image16;
source_image16.dataType_ = datatype_t::u16;
source_image16.pixels_ = image_size;
source_image16.dataPtr_ = new uint16_t[image_size];
image_library.push_back(std::move(source_image16));
auto destination = new uint16_t[image_size];
CopyImage(image_library[0], destination); // copy a 8-bit image into 16-bit destination
CopyImage(image_library[1], destination); // copy a 16-bit image into 16-bit destination
// deletes, etc to follow (not shown for conciseness)
}
What I want is a function which I can include in the <...> of the reinterpret_cast, which when called, returns the typename (i.e. uint8_t, uint16_t, etc). This function would be a member function of the Image class, refer to the dataType_ variable, and provide a typename - similar to what the switch/case statement is doing explicitly for each possible type - but only have to maintain it in once place since it's a function which would be callable from anywhere I'm trying to do the reinterpret_cast<> operation.
For example, a function called Image::ReturnType() which could be used like this:
std::copy_n(reinterpret_cast<source.ReturnType()>(source.dataPtr_), source.pixels_, dst_ptr);
and be defined something like this:
typename Image::ReturnType() {
switch (dataType_) {
case u8:
return uint8_t;
case u16:
return uint16_t;
}
}
I realize this question is begging for a solution that takes advantage of polymorphism or is related to templating the Image class so that I have separate Image<uint8_t> and Image<uint16_t> types, but this then precludes me from storing a bunch of Image objects (with different pixel representations) in a single std::vector<Image>.
Note: If there is a way to store, in a single std::vector, objects with different template type specializations, I'm also open to that solution, but I'm afraid that doesn't exist.
Many thanks!
All types must be known at compile time. A type is a compile time abstraction. At runtime, types are more or less existant.
Is your dataType_ variable is known at compile time? If yes, of course you can have a metafunction returning a type:
using TypeMap = std::tuple<uint8_t, uint16_t>;
struct Image {
// ...
template<datatype_t datatype_id>
using ReturnType = std::tuple_element_t<static_cast<std::size_t>(datatype_id), TypeMap>;
};
Then you can use the metafunction:
// the_type_to_use is uint8_t
using the_type_to_use = Image::ReturnType<datatype_t::u8>;
On the other hand, if the value is known at runtime only, then you must use a runtime distpatch. Whether it's through a variant or a virtual table.
Polymorphism works with pointers. You need a common non-templated base class for your Image<...> . You store (smart) pointers to that in your container.
class ImageBase {
...;
};
template <typename DataType>
class Image: public ImageBase {
std::vector<DataType> data_;
...;
};
std::vector<std::unique_ptr<ImageBase>> image_library;
No.
A function can only return an object or a reference (or void).
What you can do is map a value to a type using a templated type alias, or a member type alias of a class template. Example:
template<datatype_t>
struct image_return {};
template<>
image_return<u8> {
using type = std::uint8_t;
};
template<>
image_return<u16> {
using type = std::uint16_t;
};
For your use case however, instead of void*, it might be better to use std::variant<std::uint8_t*, std::uint16_t*>. Or even better, use a variant of std::vectors rather than bare pointers to allocated memory.
I have tried union...
struct foo
{
union
{
struct // 2 bytes
{
char var0_1;
};
struct // 5 bytes
{
char var1_1;
int var1_2;
};
};
};
Problem: Unions do what I want, except they will always take the size of the biggest datatype. In my case I need struct foo to have some initialization that allows me to tell it which structure to chose of the two (if that is even legal) as shown below.
So after that, I tried class template overloading...
template <bool B>
class foo { }
template <>
class foo<true>
{
char var1;
}
template <>
class foo<false>
{
char var0;
int var1;
}
Problem: I was really happy with templates and the fact that I could use the same variable name on the char and int, but the problem was the syntax. Because the classes are created on compile-time, the template boolean variable needed to be a hardcoded constant, but in my case the boolean needs to be user-defined on runtime.
So I need something of the two "worlds." How can I achieve what I'm trying to do?
!!NOTE: The foo class/struct will later be inherited, therefore as already mentioned, size of foo is of utmost importance.
EDIT#1::
Application:
Basically this will be used to read/write (using a pointer as an interface) a specific data buffer and also allow me to create (new instance of the class/struct) the same data buffer. The variables you see above specify the length. If it's a smaller data buffer, the length is written in a char/byte. If it's a bigger data buffer, the first char/byte is null as a flag, and the int specifies the length instead. After the length it's obvious that the actual data follows, hence why the inheritance. Size of class is of the utmost importance. I need to have my cake and eat it too.
A layer of abstraction.
struct my_buffer_view{
std::size_t size()const{
if (!m_ptr)return 0;
if (*m_ptr)return *m_ptr;
return *reinterpret_cast<std::uint32_t const*>(m_ptr+1);
}
std::uint8_t const* data() const{
if(!m_ptr)return nullptr;
if(*m_ptr)return m_ptr+1;
return m_ptr+5;
}
std::uint8_t const* begin()const{return data();}
std::uint8_t const* end()const{return data()+size();}
my_buffer_view(std::uint_t const*ptr=nullptr):m_ptr(ptr){}
my_buffer_view(my_buffer_view const&)=default;
my_buffer_view& operator=(my_buffer_view const&)=default;
private:
std::uint8_t const* m_ptr=0;
};
No variable sized data anywhere. I coukd have used a union for size etx:
struct header{
std::uint8_t short_len;
union {
struct{
std::uint32_t long_len;
std::uint8_t long_buf[1];
}
struct {
std::short_buf[1];
}
} body;
};
but I just did pointer arithmetic instead.
Writing such a buffer to a bytestream is another problem entirely.
Your solution does not make sense. Think about your solution: you could define two independents classes: fooTrue and fooFalse with corresponding members exactly with the same result.
Probably, you are looking for a different solution as inheritance. For example, your fooTrue is baseFoo and your fooFalse is derivedFoo with as the previous one as base and extends it with another int member.
In this case, you have the polymorphism as the method to work in runtime.
You can't have your cake and eat it too.
The point of templates is that the specialisation happens at compile time. At run time, the size of the class is fixed (albeit, in an implementation-defined manner).
If you want the choice to be made at run time, then you can't use a mechanism that determines size at compile-time. You will need a mechanism that accommodates both possible needs. Practically, that means your base class will need to be large enough to contain all required members - which is essentially what is happening with your union based solution.
In reference to your "!!NOTE". What you are doing qualifies as premature optimisation. You are trying to optimise size of a base class without any evidence (e.g. measurement of memory usage) that the size difference is actually significant for your application (e.g. that it causes your application to exhaust available memory). The fact that something will be a base for a number of other classes is not sufficient, on its own, to worry about its size.
What I have:
Vectors of different custom structs(one custom struct per vector)
I used pointers to each struct to give it a static size per record
A vector that combines these vectors via a pointer to each
When I try to cast a pointer to the custom vector it fails on every iteration
My workaround is a call to a function that takes the vector pointer as an argument and returns a void pointer.
I know this is wrong despite being functional, but I can't find a reference on the right way to define the cast method properly.
I'm looking for the right way to accomplish this.
typedef struct mystruct {
DWORD something;
vectorclassa * somethinga;
vectorclassb * somethingb;
};
typedef std::vector<mystruct*> amystruct;
void * theWrongWay(mystruct * apointer){
return apointer;
}
typedef std::vector bigvector;
If I try to bigvector.push_back(&instance_of_amystruct)
It fails. If I change std::vector<amystruct*> bigvector to a void* and call theWrongWay with a &amystruct instance, it compiles/runs. It just seems wrong.
The problem is that I don't know how to define the missing method for vector or cast it to something vector knows how to deal with without doing something...bad.
It's very hard to answer this question because it's hard to tell why you want to do this. Why do you want to keep vectors of different types in another vector?
Anyway, I'll just assume that you want to do this because there's some common action you want to take on each vector. Then you can simply define an abstract class that defines that action, and have a templated child class of this class that keep a different vector depending on the template argument. You can then keep the vectors in a container referring to them as their common ancestor. Some code:
class ActionableVector {
virtual void doSuperCoolStuff() = 0;
}
template<typename T>
class VectorHandler: public ActionableVector {
vector<T> handledVector;
// vector<T> & handleVector; // You can keep a reference to an external vector too
virtual void doSuperCoolStuff() {
//do super cool stuff in a type-safe manner
}
}
template<>
class VectorHandler<ATypeThatNeedsSpecialAttention>: public ActionableVector {
Vector<ATypeThatNeedsSpecialAttention> handledVector;
virtual void doSuperCoolStuff() {
// Do especially cool stuff
}
vector<ActionableVector*> myVectors;
for(ActionableVector * av: myVectors ) { //C++ 11 yaay
av->doSuperCoolStuff();
}
If you really really really want to keep objects of completely different types in a container, and willing to sell your soul to the devil for that, look at this answer.