I have implemented a function that resets the content of the structure to which a pointer points:
template <typename Struct>
void initialize(Struct* s)
{
*s = Struct{};
}
I have performance issues when Struct becomes big (above 10K) because Struct is created in the stack and then assigned to *s. I was wondering if I could improve it:
Is it possible to initialize directly *s without the temporary Struct{} object?
Should I instead evaluate the size of Struct and build it in the heap if it is big?
Thank you in advance
Firstly, you should probably use a reference; not a pointer. The point of this is to avoid null indirection bugs.
If the class is trivial and value initialise to zero (which is typically the case for most trivial types), then the optimiser should compile your function into a call to memset, without any need for initialisation of a temporary object. So there should be no reason to worry in that case.
You could call memset explicitly, although that is technically not portable to exotic systems in case the class contains certain types (for example, null pointer does not necessarily have the representation of zero).
Is it possible to initialize directly *s without the temporary Struct{} object?.
Yes, if you're willing to change the requirements of the function. Currently it works for classes that are default constructible and move assignable.
You can avoid creation of a temporary object if you modify the pointed object directly. In following example, there are no temporaries of type Struct created:
constexpr void
initialize(Struct& s)
{
s.member1 = T1{};
s.member2 = T2{};
To make this generic, the operation could be performed in a member function. Thus, you could specify a requirement that the pointed class has a member function with particular name and no parameters:
s.clear();
You can combine both approaches for types which they apply to:
template<class Struct>
constexpr void
initialize(Struct& s)
{
if constexpr (std::is_trivially_copyable_v<Struct>) {
// alternative 1, if you trust your optimiser
s = Struct{};
// alternative 2, if you doubt the quality of the optimiser
// technically may have different meaning on exotic systems
std::memset(&s, 0, sizeof s);
} else {
s.clear();
}
}
If you need this to work with some classes that conforms to neither requirement, then you'll need to specialise the template.
Should I instead evaluate the size of Struct and build it in the heap if it is big [10K]?
You generally should avoid having public classes that large entirely. If you need such large storage, you could wrap it in a type that allocates it dynamically. Something like this:
class Struct
{
private:
struct VeryLarge{/.../};
std::unique_ptr<VeryLarge> storage;
public:
// public interface
I am looking for a way to extend an int variable by some status information, which is only very rarely used and most of the time its just the int value that counts. This is why i would like to avoid using a struct with additional boolean members.
Relevant additional properties would be maximal 5 to 6 additional properties for example NULL, UNDEFINED, NAN, MISSING of which only one can be true.
Ideas:
An int pointer would be one way to do this for one additional property: nullptr is the one additional value the int variable can take. But in this case I could not have more exceptional states.
Another option would be to use some magic values which I do not expect to be used such as UNDEFINED = std::numeric_limits<int>::min() and MISSING = std::numeric_limits<int>::min()+1 and so on.
Question: Is there a better way to do this (which does require minimal additional memory)?
Sub-Question: Is there a good way to do this without using additional memory if I can decide if the value is of the exeptional cases during compile time?
The best available alternative is not to use ints, but some other type – your suggestion to use a int* also falls in this category. This wrapper type can of course overload various operators for convenience of accessing the underlying integer value. Defining a new type (or leveraging existing standard library types) is optimal terms of memory, as you will probably only double the size of each integer (e.g. now you might have integer + enum (backed by int or smaller) + possibly padding). Depending on your integer size, even a pointer might have more memory overhead.
If only a few integers have an extra status AND those integers will not be copied or moved, then you could store the status externally, in a map from integer identity to status. I.e. we use the pointer to an integer object as key. Per integer with an extra status this map will have a much higher memory overhead than alternatives, but depending on your usage patterns this might be the most compact solution. Clearly, there is an opportunity for memory leaks here, so you should probably wrap your integers in a custom type that that removes any map entry on destruction. Roughly like this:
enum class IntStatus { IS_NAN, IS_MISSING };
class IntWithExternalStatus {
public:
explicit IntWithExternalStatus(int x = 0) : m_value{x} {}
explicit IntWithExternalStatus(IntStatus s) : m_value{} { s_status.insert({this, s}); }
~IntWithExternalStatus() { m_status.erase(this); }
operator int& () { return m_value; }
operator int () const { return m_value; }
bool is_valid() const { return s_status.find(this) == s_status.end(); }
bool is_nan() const {
auto it = s_status.find(this);
return it != s_status.end() && it->second == IntStatus::IS_NAN;
}
bool is_missing() const {
auto it = s_status.find(this);
return it != s_status.end() && it->second == IntStatus::IS_MISSING;
}
private:
static std::unordered_map<IntWithExternalStatus const*, IntStatus> s_status;
int m_value;
};
Maybe all of these extra types are unnecessary overcomplication. If you only have a few integer variables that need an extra status, creating a separate variable for the status may be easiest. E.g.:
int m_foo;
int m_bar;
IntStatus m_foo_status;
IntStatus m_bar_status;
Due to alignment issues, this might result in a more compact memory layout than defining a combined int-and-status object.
Using special values for your statuses is an easy solution without space overhead, but has a huge disadvantage: any arithmetic on those integers will erase the status and produce bogus values. You will need lots of runtime checks to prevent that, which would be best encapsulated in a separate type.
Regarding the compile-time subquestion, this depends on your definition of memory use. E.g. you could use template metaprogramming that selects an int or InvalidInt type depending on the compile-time value, where an InvalidInt is a kind of null-object pattern. However, this will generate specialized code for all instantiated templates. In particular, the code using your status-ints would also have to be templated. This could reduce total memory use if you have lots of status-ints at the same time that all have the same status, but is unlikely to be helpful in other scenarios.
In C++, is it possible to define a sort order for pointers to member functions? It seems that the operator< is undefined. Also, it's illegal to cast to void*.
class A
{
public:
void Test1(){}
void Test2(){}
};
int main()
{
void (A::* const one)() = &A::Test1;
void (A::* const two)() = &A::Test2;
bool equal = one == two; //Equality works fine.
bool less = one < two; //Less than doesn't.
return 0;
}
Thanks!
Function pointers are not relationally comparable in C++. Equality comparisons are supported, except for situations when at least one of the pointers actually points to a virtual member function (in which case the result is unspecified).
Of course, you can always introduce an ordering by implementing a comparison predicate and comparing the pointers explicitly (won't look too elegant though, since you can only use equality comparisons). Other possible solutions would cross into the territory of the various implementation-specific "hacks".
Member function pointers are not actual pointers. You should look at them as opaque structs. What does a method pointer contain:
struct method_pointer {
bool method_is_virtual;
union {
unsigned vtable_offset; // for a virtual function, need the vtable entry
void* function_pointer; // otherwise need the pointer to the concrete method
}
};
If you could cast this to void* (you can't) all you would have is a pointer the the struct, not a pointer to code. That's why operator<() is undefined as well since the value of the struct's pointer is just where ever it happens to be in memory.
In addition to that, what are you sorting by?
How can i hash (std::tr1::hash or boost::hash) a c++ pointer-to-member-function?
Example:
I have several bool (Class::*functionPointer)() (not static) that point to several diferent methods of the class Class and i need to hash those pointer-to-member-function.
How can i do that?
Also how can i compare (std::less) those member function pointers so i can store them in a std::set?
All C++ objects, including pointers to member functions, are represented in memory as an array of chars. So you could try:
bool (Class::*fn_ptr)() = &Class::whatever;
const char *ptrptr = static_cast<const char*>(static_cast<const void*>(&fn_ptr));
Now treat ptrptr as pointing to an array of (sizeof(bool (Class::*)())) bytes, and hash or compare those bytes. You can use unsigned char instead of char if you prefer.
This guarantees no false positives - in C++03, pointers to member functions are POD, which means among other things that they can be copied using memcpy. This implies that if have the same byte-for-byte values, then they are the same.
The problem is that the storage representation of member function pointers could include bits which do not participate in the value - so they will not necessarily be the same for different pointers to the same member function. Or the compiler might, for some obscure reason, have more than one way of pointing to the same function of the same class, which are not byte-wise equal. Either way you can get false negatives. You'll have to look into how member function pointers actually work on your implementation. It must implement operator== for member function pointers somehow, and if you can find out how then you can probably figure out an order and a hash function.
That's potentially hard: member function pointers are awkward, and the storage is likely to include different amounts of non-participating "slack space" according to what kind of function is pointed to (virtual, inherited). So you'll probably have to interact quite significantly with your compiler's implementation details. This article might help get you started: http://www.codeproject.com/KB/cpp/FastDelegate.aspx
A cleaner alternative might be to do a linear search through an array in order to "canonicalise" all your function pointers, then compare and hash based on the position of the "canonical" instance of that function pointer in your array. Depends what your performance requirements are. And even if there are requirements, does the class (and its derived classes) have so many functions that the linear search will take that long?
typedef bool (Class::*func)();
vector<func> canon;
size_t getIndexOf(func fn_ptr) {
vector<func>::iterator it = find(canon.begin(), canon.end(), fn_ptr);
if (it != canon.end()) return it - canon.begin();
canon.push_back(func);
return canon.size() - 1;
}
I could not cast the pointer (in Microsoft compiler 2010)as described in previous answer but this works for me:
static string fmptostr(int atype::*opt)
{
char buf[sizeof(opt)];
memcpy(&buf,&opt,sizeof(opt));
return string(buf,sizeof(opt));
}
About bitwise identity of the pointer, it can be bitwise so it seems if appropriate compiler switches are used. At least this is true for Microsoft compiler E.g
using #pragma pointers_to_members
and a switch.../vmg
If your member function pointer is unique, which is true in most of cases for callback-based subscriptions, then you can use the tick with type_index, which uniqueness is guaranteed by uniqueness of type (i.e. Class::Method) in your program, and it is suitable to be stored in unordered_map, i.e.
struct MyEvent {
using fn_t = std::function<void(MyEvent &)>;
using map_t = std::unordered_map<std::type_index, fn_t>;
template <typename Handler>
void subscribe(Object& obj, Handler&& handler) {
fn_t fn = [&, handler = std::move(handler)](MyEvent& event) {
(obj.*handler)(event);
}
std::type_index index = typeid(Handler);
subscribers.emplace(std::move(index), std::move(fn));
}
void fire() {
for(auto& pair: subscribers) {
auto& fn = pair.second;
fn(*this);
}
}
map_t subscribers;
}
And the subscription and fire event example:
MyEvent event;
MyObject obj = ...;
event.subscribe(obj, &MyObject::on_event );
...
event.fire();
So, example above gives you class/method uniqueness, and if you need object/method uniqueness, then you should have an struct, which provides combined hash, assuming that there is std::hash<MyObject> and there is already std::hash<std::type_index> for a member function pointer.
Is this the best way to make a variable sized struct in C++? I don't want to use vector because the length doesn't change after initialization.
struct Packet
{
unsigned int bytelength;
unsigned int data[];
};
Packet* CreatePacket(unsigned int length)
{
Packet *output = (Packet*) malloc((length+1)*sizeof(unsigned int));
output->bytelength = length;
return output;
}
Edit: renamed variable names and changed code to be more correct.
Some thoughts on what you're doing:
Using the C-style variable length struct idiom allows you to perform one free store allocation per packet, which is half as many as would be required if struct Packet contained a std::vector. If you are allocating a very large number of packets, then performing half as many free store allocations/deallocations may very well be significant. If you are also doing network accesses, then the time spent waiting for the network will probably be more significant.
This structure represents a packet. Are you planning to read/write from a socket directly into a struct Packet? If so, you probably need to consider byte order. Are you going to have to convert from host to network byte order when sending packets, and vice versa when receiving packets? If so, then you could byte-swap the data in place in your variable length struct. If you converted this to use a vector, it would make sense to write methods for serializing / deserializing the packet. These methods would transfer it to/from a contiguous buffer, taking byte order into account.
Likewise, you may need to take alignment and packing into account.
You can never subclass Packet. If you did, then the subclass's member variables would overlap with the array.
Instead of malloc and free, you could use Packet* p = ::operator new(size) and ::operator delete(p), since struct Packet is a POD type and does not currently benefit from having its default constructor and its destructor called. The (potential) benefit of doing so is that the global operator new handles errors using the global new-handler and/or exceptions, if that matters to you.
It is possible to make the variable length struct idiom work with the new and delete operators, but not well. You could create a custom operator new that takes an array length by implementing static void* operator new(size_t size, unsigned int bitlength), but you would still have to set the bitlength member variable. If you did this with a constructor, you could use the slightly redundant expression Packet* p = new(len) Packet(len) to allocate a packet. The only benefit I see compared to using global operator new and operator delete would be that clients of your code could just call delete p instead of ::operator delete(p). Wrapping the allocation/deallocation in separate functions (instead of calling delete p directly) is fine as long as they get called correctly.
If you never add a constructor/destructor, assignment operators or virtual functions to your structure using malloc/free for allocation is safe.
It's frowned upon in c++ circles, but I consider the usage of it okay if you document it in the code.
Some comments to your code:
struct Packet
{
unsigned int bitlength;
unsigned int data[];
};
If I remember right declaring an array without a length is non-standard. It works on most compilers but may give you a warning. If you want to be compliant declare your array of length 1.
Packet* CreatePacket(unsigned int length)
{
Packet *output = (Packet*) malloc((length+1)*sizeof(unsigned int));
output->bitlength = length;
return output;
}
This works, but you don't take the size of the structure into account. The code will break once you add new members to your structure. Better do it this way:
Packet* CreatePacket(unsigned int length)
{
size_t s = sizeof (Packed) - sizeof (Packed.data);
Packet *output = (Packet*) malloc(s + length * sizeof(unsigned int));
output->bitlength = length;
return output;
}
And write a comment into your packet structure definition that data must be the last member.
Btw - allocating the structure and the data with a single allocation is a good thing. You halve the number of allocations that way, and you improve the locality of data as well. This can improve the performance quite a bit if you allocate lots of packages.
Unfortunately c++ does not provide a good mechanism to do this, so you often end up with such malloc/free hacks in real world applications.
This is OK (and was standard practice for C).
But this is not a good idea for C++.
This is because the compiler generates a whole set of other methods automatically for you around the class. These methods do not understand that you have cheated.
For Example:
void copyRHSToLeft(Packet& lhs,Packet& rhs)
{
lhs = rhs; // The compiler generated code for assignement kicks in here.
// Are your objects going to cope correctly??
}
Packet* a = CreatePacket(3);
Packet* b = CreatePacket(5);
copyRHSToLeft(*a,*b);
Use the std::vector<> it is much safer and works correctly.
I would also bet it is just as efficient as your implementation after the optimizer kicks in.
Alternatively boost contains a fixed size array:
http://www.boost.org/doc/libs/1_38_0/doc/html/array.html
You can use the "C" method if you want but for safety make it so the compiler won't try to copy it:
struct Packet
{
unsigned int bytelength;
unsigned int data[];
private:
// Will cause compiler error if you misuse this struct
void Packet(const Packet&);
void operator=(const Packet&);
};
I'd probably just stick with using a vector<> unless the minimal extra overhead (probably a single extra word or pointer over your implementation) is really posing a problem. There's nothing that says you have to resize() a vector once it's been constructed.
However, there are several The advantages of going with vector<>:
it already handles copy, assignment & destruction properly - if you roll your own you need to ensure you handle these correctly
all the iterator support is there - again, you don't have to roll your own.
everybody already knows how to use it
If you really want to prevent the array from growing once constructed, you might want to consider having your own class that inherits from vector<> privately or has a vector<> member and only expose via methods that just thunk to the vector methods those bits of vector that you want clients to be able to use. That should help get you going quickly with pretty good assurance that leaks and what not are not there. If you do this and find that the small overhead of vector is not working for you, you can reimplement that class without the help of vector and your client code shouldn't need to change.
There are already many good thoughts mentioned here. But one is missing. Flexible Arrays are part of C99 and thus aren't part of C++, although some C++ compiler may provide this functionality there is no guarantee for that. If you find a way to use them in C++ in an acceptable way, but you have a compiler that doesn't support it, you perhaps can fallback to the "classical" way
If you are truly doing C++, there is no practical difference between a class and a struct except the default member visibility - classes have private visibility by default while structs have public visibility by default. The following are equivalent:
struct PacketStruct
{
unsigned int bitlength;
unsigned int data[];
};
class PacketClass
{
public:
unsigned int bitlength;
unsigned int data[];
};
The point is, you don't need the CreatePacket(). You can simply initialize the struct object with a constructor.
struct Packet
{
unsigned long bytelength;
unsigned char data[];
Packet(unsigned long length = 256) // default constructor replaces CreatePacket()
: bytelength(length),
data(new unsigned char[length])
{
}
~Packet() // destructor to avoid memory leak
{
delete [] data;
}
};
A few things to note. In C++, use new instead of malloc. I've taken some liberty and changed bitlength to bytelength. If this class represents a network packet, you'll be much better off dealing with bytes instead of bits (in my opinion). The data array is an array of unsigned char, not unsigned int. Again, this is based on my assumption that this class represents a network packet. The constructor allows you to create a Packet like this:
Packet p; // default packet with 256-byte data array
Packet p(1024); // packet with 1024-byte data array
The destructor is called automatically when the Packet instance goes out of scope and prevents a memory leak.
You probably want something lighter than a vector for high performances. You also want to be very specific about the size of your packet to be cross-platform. But you don't want to bother about memory leaks either.
Fortunately the boost library did most of the hard part:
struct packet
{
boost::uint32_t _size;
boost::scoped_array<unsigned char> _data;
packet() : _size(0) {}
explicit packet(packet boost::uint32_t s) : _size(s), _data(new unsigned char [s]) {}
explicit packet(const void * const d, boost::uint32_t s) : _size(s), _data(new unsigned char [s])
{
std::memcpy(_data, static_cast<const unsigned char * const>(d), _size);
}
};
typedef boost::shared_ptr<packet> packet_ptr;
packet_ptr build_packet(const void const * data, boost::uint32_t s)
{
return packet_ptr(new packet(data, s));
}
There's nothing whatsoever wrong with using vector for arrays of unknown size that will be fixed after initialization. IMHO, that's exactly what vectors are for. Once you have it initialized, you can pretend the thing is an array, and it should behave the same (including time behavior).
Disclaimer: I wrote a small library to explore this concept: https://github.com/ppetr/refcounted-var-sized-class
We want to allocate a single block of memory for a data structure of type T and an array of elements of type A. In most cases A will be just char.
For this let's define a RAII class to allocate and deallocate such a memory block. This poses several difficulties:
C++ allocators don't provide such API. Therefore we need to allocate plain chars and place the structure in the block ourselves. For this std::aligned_storage will be helpful.
The memory block must be properly aligned. Because in C++11 there doesn't seem to be API for allocating an aligned block, we need to slightly over-allocate by alignof(T) - 1 bytes and then use std::align.
// Owns a block of memory large enough to store a properly aligned instance of
// `T` and additional `size` number of elements of type `A`.
template <typename T, typename A = char>
class Placement {
public:
// Allocates memory for a properly aligned instance of `T`, plus additional
// array of `size` elements of `A`.
explicit Placement(size_t size)
: size_(size),
allocation_(std::allocator<char>().allocate(AllocatedBytes())) {
static_assert(std::is_trivial<Placeholder>::value);
}
Placement(Placement const&) = delete;
Placement(Placement&& other) {
allocation_ = other.allocation_;
size_ = other.size_;
other.allocation_ = nullptr;
}
~Placement() {
if (allocation_) {
std::allocator<char>().deallocate(allocation_, AllocatedBytes());
}
}
// Returns a pointer to an uninitialized memory area available for an
// instance of `T`.
T* Node() const { return reinterpret_cast<T*>(&AsPlaceholder()->node); }
// Returns a pointer to an uninitialized memory area available for
// holding `size` (specified in the constructor) elements of `A`.
A* Array() const { return reinterpret_cast<A*>(&AsPlaceholder()->array); }
size_t Size() { return size_; }
private:
// Holds a properly aligned instance of `T` and an array of length 1 of `A`.
struct Placeholder {
typename std::aligned_storage<sizeof(T), alignof(T)>::type node;
// The array type must be the last one in the struct.
typename std::aligned_storage<sizeof(A[1]), alignof(A[1])>::type array;
};
Placeholder* AsPlaceholder() const {
void* ptr = allocation_;
size_t space = sizeof(Placeholder) + alignof(Placeholder) - 1;
ptr = std::align(alignof(Placeholder), sizeof(Placeholder), ptr, space);
assert(ptr != nullptr);
return reinterpret_cast<Placeholder*>(ptr);
}
size_t AllocatedBytes() {
// We might need to shift the placement of for up to `alignof(Placeholder) - 1` bytes.
// Therefore allocate this many additional bytes.
return sizeof(Placeholder) + alignof(Placeholder) - 1 +
(size_ - 1) * sizeof(A);
}
size_t size_;
char* allocation_;
};
Once we've dealt with the problem of memory allocation, we can define a wrapper class that initializes T and an array of A in an allocated memory block.
template <typename T, typename A = char,
typename std::enable_if<!std::is_destructible<A>{} ||
std::is_trivially_destructible<A>{},
bool>::type = true>
class VarSized {
public:
// Initializes an instance of `T` with an array of `A` in a memory block
// provided by `placement`. Callings a constructor of `T`, providing a
// pointer to `A*` and its length as the first two arguments, and then
// passing `args` as additional arguments.
template <typename... Arg>
VarSized(Placement<T, A> placement, Arg&&... args)
: placement_(std::move(placement)) {
auto [aligned, array] = placement_.Addresses();
array = new (array) char[placement_.Size()];
new (aligned) T(array, placement_.Size(), std::forward<Arg>(args)...);
}
// Same as above, with initializing a `Placement` for `size` elements of `A`.
template <typename... Arg>
VarSized(size_t size, Arg&&... args)
: VarSized(Placement<T, A>(size), std::forward<Arg>(args)...) {}
~VarSized() { std::move(*this).Delete(); }
// Destroys this instance and returns the `Placement`, which can then be
// reused or destroyed as well (deallocating the memory block).
Placement<T, A> Delete() && {
// By moving out `placement_` before invoking `~T()` we ensure that it's
// destroyed even if `~T()` throws an exception.
Placement<T, A> placement(std::move(placement_));
(*this)->~T();
return placement;
}
T& operator*() const { return *placement_.Node(); }
const T* operator->() const { return &**this; }
private:
Placement<T, A> placement_;
};
This type is moveable, but obviously not copyable. We could provide a function to convert it into a shared_ptr with a custom deleter. But this will need to internally allocate another small block of memory for a reference counter (see also How is the std::tr1::shared_ptr implemented?).
This can be solved by introducing a specialized data type that will hold our Placement, a reference counter and a field with the actual data type in a single structure. For more details see my refcount_struct.h.
You should declare a pointer, not an array with an unspecified length.