Consider the following, simplified and incomplete, implementation of a fixed-sized vector:
template<typename T>
class Vec {
T *start, *end;
public:
T& operator[](ssize_t idx) { return start[idx]; }
void pop() {
end--;
end->~T();
}
template<typename... U>
void push(U... args) {
new (end) T { std::forward<U>(args)... };
end++;
}
};
Now consider the following T:
struct T {
const int i;
};
And the following use case:
Vec<T> v;
v.push(1);
std::cout << v[0].i;
v.pop();
v.push(2);
std::cout << v[0].i;
The index operator uses the start pointer to access the object. The object at that point was destroyed by pop and another object was created in its storage location by push(2). If I read the documentation surrounding std::launder correctly, this means that the behavior of v[0] in the line below is undefined.
How is std::launder supposed to be used to correct this code? Do we have to launder start and end each time placement new is used? Current implementations of the stdlib seem to be using code similar to the one posted above. Is the behavior of these implementations undefined?
How is std::launder supposed to be used to correct this code? Do we have to launder start and end each time placement new is used?
From P0532R0, you could avoid needing to call launder() if the return value of placement new is assigned to end. You would not need to change your start pointer unless the vector was empty since the object currently pointed to by start would still have an active lifetime with the code you provided.
The same paper indicates that launder() is a no-op unless the object lifetime has ended and has been replaced with a new object, so using launder() will not incur a performance penalty if it is unnecessary:
[...] the type of std::launder(this) is equivalent to just this as Richard Smith pointed out: Remember that launder(p) is a no-op unless p points to an object whose lifetime has ended and where a new object has been created in the same storage.
Current implementations of the stdlib seem to be using code similar to the one posted above. Is the behavior of these implementations undefined?
Yes. P0532R0 also discusses this issue and the content is similar to the discussion in the question's comments: vector does not use placement new directly, the return value of the placement new call is lost in the chain of function calls to the vector's allocator, and in any event placement new is used element by element so constructing the internal vector machinery cannot use the return value anyway. launder() appears to be the tool intended to be used here. However, the pointer type specified by the allocator is not required to be a raw pointer type at all and launder() only works for raw pointers. The current implementation is currently undefined for some types; launder() does not seem to be the appropriate machinery for solving the generic case for allocator based containers.
Related
We are initializing (large) arrays of trivially_copiable objects from secondary storage, and questions such as this or this leaves us with little confidence in our implemented approach.
Below is a minimal example to try to illustrate the "worrying" parts in the code.
Please also find it on Godbolt.
Example
Let's have a trivially_copyable but not default_constructible user type:
struct Foo
{
Foo(double a, double b) :
alpha{a},
beta{b}
{}
double alpha;
double beta;
};
Trusting cppreference:
Objects of trivially-copyable types that are not potentially-overlapping subobjects are the only C++ objects that may be safely copied with std::memcpy or serialized to/from binary files with std::ofstream::write()/std::ifstream::read().
Now, we want to read a binary file into an dynamic array of Foo. Since Foo is not default constructible, we cannot simply:
std::unique_ptr<Foo[]> invalid{new Foo[dynamicSize]}; // Error, no default ctor
Alternative (A)
Using uninitialized unsigned char array as storage.
std::unique_ptr<unsigned char[]> storage{
new unsigned char[dynamicSize * sizeof(Foo)] };
input.read(reinterpret_cast<char *>(storage.get()), dynamicSize * sizeof(Foo));
std::cout << reinterpret_cast<Foo *>(storage.get())[index].alpha << "\n";
Is there an UB because object of actual type Foo are never explicitly created in storage?
Alternative (B)
The storage is explicitly typed as an array of Foo.
std::unique_ptr<Foo[]> storage{
static_cast<Foo *>(::operator new[](dynamicSize * sizeof(Foo))) };
input.read(reinterpret_cast<char *>(storage.get()), dynamicSize * sizeof(Foo));
std::cout << storage[index].alpha << "\n";
This alternative was inspired by this post. Yet, is it better defined? It seems there are still no explicit creation of object of type Foo.
It is notably getting rid of the reinterpret_cast when accessing the Foo data member (this cast might have violated the Type Aliasing rule).
Overall Questions
Are any of these alternatives defined by the standard? Are they actually different?
If not, is there a correct way to implement this (without first initializing all Foo instances to values that will be discarded immediately after)
Is there any difference in undefined behaviours between versions of the C++ standard?
(In particular, please see this comment with regard to C++20)
What you're trying to do ultimately is create an array of some type T by memcpying bytes from elsewhere without default constructing the Ts in the array first.
Pre-C++20 cannot do this without provoking UB at some point.
The problem ultimately comes down to [intro.object]/1, which defines the ways objects get created:
An object is created by a definition, by a new-expression, when implicitly changing the active member of a union, or when a temporary object is created ([conv.rval], [class.temporary]).
If you have a pointer of type T*, but no T object has been created in that address, you can't just pretend that the pointer points to an actual T. You have to cause that T to come into being, and that requires doing one of the above operations. And the only available one for your purposes is the new-expression, which requires that the T is default constructible.
If you want to memcpy into such objects, they must exist first. So you have to create them. And for arrays of such objects, that means they need to be default constructible.
So if it is at all possible, you need a (likely defaulted) default constructor.
In C++20, certain operations can implicitly create objects (provoking "implicit object creation" or IOC). IOC only works on implicit lifetime types, which for classes:
A class S is an implicit-lifetime class if it is an aggregate or has at least one trivial eligible constructor and a trivial, non-deleted destructor.
Your class qualifies, as it has a trivial copy constructor (which is "eligible") and a trivial destructor.
If you create an array of byte-wise types (unsigned char, std::byte, or char), this is said to "implicitly create objects" in that storage. This property also applies to the memory returned by malloc and operator new. This means that if you do certain kinds of undefined behavior to pointers to that storage, the system will automatically create objects (at the point where the array was created) that would make that behavior well-defined.
So if you allocate such storage, cast a pointer to it to a T*, and then start using it as though it pointed to a T, the system will automatically create Ts in that storage, so long as it was appropriately aligned.
Therefore, your alternative A works just fine:
When you apply [index] to your casted pointer, C++ will retroactively create an array of Foo in that storage. That is, because you used the memory like an array of Foo exists there, C++20 will make an array of Foo exist there, exactly as if you had created it back at the new unsigned char statement.
However, alternative B will not work as is. You did not use new[] Foo to create the array, so you cannot use delete[] Foo to delete it. You can still use unique_ptr, but you'll have to create a deleter that explicitly calls operator delete on the pointer:
struct mem_delete
{
template<typename T>
void operator(T *ptr)
{
::operator delete[](ptr);
}
};
std::unique_ptr<Foo[], mem_delete> storage{
static_cast<Foo *>(::operator new[](dynamicSize * sizeof(Foo))) };
input.read(reinterpret_cast<char *>(storage.get()), dynamicSize * sizeof(Foo));
std::cout << storage[index].alpha << "\n";
Again, storage[index] creates an array of T as if it were created at the time the memory was allocated.
My first question is: What are you trying to achieve?
Is there an issue with reading each entry individually?
Are you assuming that your code will speed up by reading an array?
Is latency really a factor?
Why can't you just add a default constructor to the class?
Why can't you enhance input.read() to read directly into an array? See std::extent_v<T>
Assuming the constraints you defined, I would start with writing it the simple way, reading one entry at a time, and benchmark it.
Having said that, that which you describe is a common paradigm and, yes, can break a lot of rules.
C++ is very (overly) cautious about things like alignment which can be issues on certain platforms and non-issues on others. This is only "undefined behaviour" because no cross-platform guarantees can be given by the C++ standard itself, even though many techniques work perfectly well in practice.
The textbook way to do this is to create an empty buffer and memcpy into a proper object, but as your input is serialised (potentially by another system), there isn't actually a guarantee that the padding and alignment will match the memory layout which the local compiler determined for the sequence so you would still have to do this one item at a time.
My advice is to write a unit-test to ensure that there are no issues and potentially embed that into the code as a static assertion. The technique you described breaks some C++ rules but that doesn't mean it's breaking, for example, x86 rules.
Alternative (A): Accessing a —non-static— member of an object before its lifetime begins.
The behavior of the program is undefined (See: [basic.life]).
Alternative (B): Implicit call to the implicitly deleted default constructor.
The program is ill-formed (See: [class.default.ctor]).
I'm not sure about the latter. If someone more knowledgeable knows if/why this is UB please correct me.
You can manage the memory yourself, and then return a unique_ptr which uses a custom deleter. Since you can't use new[], you can't use the plain version of unique_ptr<T[]> and you need to manually call the destructor and deleter using an allocator.
template <class Allocator = std::allocator<Foo>>
struct FooDeleter : private Allocator {
using pointer = typename std::allocator_traits<Allocator>::pointer;
explicit FooDeleter(const Allocator &alloc, len) : Allocator(alloc), len(len) {}
void operator()(pointer p) {
for (pointer i = p; i != p + len; ++i) {
Allocator::destruct(i);
}
Allocator::deallocate(p, len);
}
size_t len;
};
std::unique_ptr<Foo[], FooDeleter<>> create(size_t len) {
std::allocator<Foo> alloc;
Foo *p = nullptr, *i = nullptr;
try {
p = alloc.allocate(len);
for (i = p; i != p + len; ++i) {
alloc.construct(i , 1.0f, 2.0f);
}
} catch (...) {
while (i > p) {
alloc.destruct(i--);
}
if (p)
alloc.deallocate(p);
throw;
}
return std::unique_ptr<Foo[], FooDeleter<>>{p, FooDeleter<>(alloc, len)};
}
I'm studying C++ and in all my "experiments" i'm trying to understand temporary objects (rvalues) lifetime.
My question is:
Having an object which contains a const char* pointer, what happens when i want to use a constructor which takes "const char*" as argument? Usually temporary objects get destructed automatically but what happens with pointers created in this way?
I'm not using std::string or other c++11 classes for now because I'm just focusing on understanding rvalueness.
An example below:
class MyAwesomeClass {
private:
const char* data;
public:
MyAwesomeClass(const char* ptr) {
this->data = ptr;
}
MyAwesomeClass(MyAwesomeClass&& myAwesomeClassRVALUE) {
this->data = myAwesomeClassRVALUE.data;
myAwesomeClassRVALUE.data = nullptr;
}
~MyAwesomeClass() {
delete data;
}
};
int main() {
MyAwesomeClass s = "My Awesome Class' string data.";
return 0;
}
Usually temporary objects get destructed automatically but what happens with pointers created in this way?
Pointers are objects themselves. Same thing happens to temporary pointer objects as happens to all other temporary objects.
Your example deletes a pointer pointing to a string literal, so the behaviour of the program is undefined. The class would also have a broken assignement operators even if it was used correctly by passing pointer to an array allocated with new[].
As for rvalueness, the only rvalue expression in the example that I could find is nullptr.
Note this answer is based on a previous edit of the question where MyAwesomeClass was a String class.
Your String class is not really a string class as it doesn't own the underlying string data. It's more akin to std::string_view.
You have two avenues I can see you can pursue:
Your class owns the underlying data. In this case the class is a wrapper around std::string and has std::stringdata member. Look up composition. No need to worry about lifetime asstd::string` is well behaved.
Your class is like a "pointer/reference/view" to another string. In this case you have a const char* and maybe std::size_t size data member. You basically have a std::string_view (except for the wisdom, expertise and experience that went into designing std::string_view). Since you don't own the underlying data you can't do anything about the lifetime of the underlying data. The user of the class must make sure it doesn't end with a "YourStringView" to an expired object, just as he/she needs to make sure it doesn't end up with a reference/pointer to an expired object.
The semantics of these two scenarios are wildly different (as the difference between an object and a pointer to an object).
Anyway I wouldn't recommend you do any of this except for maybe learning reasons. std::string_view already exists so just use that. If you want the printing capabilities use the fmt library or the C++ format library (that is based on the mentioned fmt library).
Even if you decide to do this for learning purposes I highly encourage you look into these alternatives and learn from how they are doing things.
MyAwesomeClass(const char* ptr)
{
this->data = ptr;
}
~MyAwesomeClass()
{
delete data;
}
Oh no, no, no! No!!
Please look into RAII and rule of 0/3/5. Your class either owns the pointed object or it doesn't. If it owns it then it is responsible for creating it and deleting it. If it doesn't then it can't do either. You can't have "half of responsibilities" where you are responsible for deleting it but not for creating it.
In user code you should never need to manually manage memory. Use the rule of 0.
Quick answer is that your class does not own the data, but just the raw pointer. Under certain conditions you will see a problem with delete operator. Raw pointer are not great tool to ensure correct object ownership.
My teammates are writing a fixed-size implementation of std::vector for a safety-critical application. We're not allowed to use heap allocation, so they created a simple array wrapper like this:
template <typename T, size_t NUM_ITEMS>
class Vector
{
public:
void push_back(const T& val);
...more vector methods
private:
// Internal storage
T storage_[NUM_ITEMS];
...implementation
};
A problem we encountered with this implementation is that it requires elements present default constructors (which is not a requirement of std::vector and created porting difficulties). I decided to hack on their implementation to make it behave more like std::vector and came up with this:
template <typename T, size_t NUM_ITEMS>
class Vector
{
public:
void push_back(const T& val);
...more vector methods
private:
// Internal storage
typedef T StorageType[NUM_ITEMS];
alignas(T) char storage_[NUM_ITEMS * sizeof(T)];
// Get correctly typed array reference
StorageType& get_storage() { return reinterpret_cast<T(&)[NUM_ITEMS]>(storage_); }
const StorageType& get_storage() const { return reinterpret_cast<const T(&)[NUM_ITEMS]>(storage_); }
};
I was then able to just search and replace storage_ with get_storage() and everything worked. An example implementation of push_back might then look like:
template <typename T, size_t NUM_ITEMS>
void Vector<T, NUM_ITEMS>::push_back(const T& val)
{
get_storage()[size_++] = val;
}
In fact, it worked so easily that it got me thinking.. Is this a good/safe use of reinterpret_cast? Is the code directly above a suitable alternative to placement new, or are there risks associated with copy/move assignment to an uninitialized object?
EDIT: In response to a comment by NathanOliver, I should add that we cannot use the STL, because we cannot compile it for our target environment, nor can we certify it.
The code you've shown is only safe for POD types (Plain Old Data), where the object's representation is trivial and thus assignment to an unconstructed object is ok.
If you want this to work in all generality (which i assume you do due to using a template), then for a type T it is undefined behavior to use the object prior to construction it. That is, you must construct the object before your e.g. assignment to that location. That means you need to call the constructor explicitly on demand. The following code block demonstrates an example of this:
template <typename T, size_t NUM_ITEMS>
void Vector<T, NUM_ITEMS>::push_back(const T& val)
{
// potentially an overflow test here
// explicitly call copy constructor to create the new object in the buffer
new (reinterpret_cast<T*>(storage_) + size_) T(val);
// in case that throws, only inc the size after that succeeds
++size_;
}
The above example demonstrates placement new, which takes the form new (void*) T(args...). It calls the constructor but does not actually perform an allocation. The visual difference is the inclusion of the void* argument to operator new itself, which is the address of the object to act on and call the constructor for.
And of course when you remove an element you'll need to destroy that explicitly as well. To do this for a type T, simply call the pseudo-method ~T() on the object. Under templated context the compiler will work out what this means, either an actual destructor call, or no-op for e.g. int or double. This is demonstrated below:
template<typename T, size_t NUM_ITEMS>
void Vector<T, NUM_ITEMS>::pop_back()
{
if (size_ > 0) // safety test, you might rather this throw, idk
{
// explicitly destroy the last item and dec count
// canonically, destructors should never throw (very bad)
reinterpret_cast<T*>(storage_)[--size_].~T();
}
}
Also, I would avoid returning a refernce to an array in your get_storage() method, as it has length information and would seem to imply that all elements are valid (constructed) objects, which of course they're not. I suggest you provide methods for getting a pointer to the start of the contiguous array of constructed objects, and another method for getting the number of constructed objects. These are the .data() and .size() methods of e.g. std::vector<T>, which would make use of your class less jarring to seasoned C++ users.
Is this a good/safe use of reinterpret_cast?
Is the code directly above a suitable alternative to placement new
No. No.
or are there risks associated with copy/move assignment to an uninitialized object?
Yes. The behaviour is undefined.
Assuming memory is uninitialised, copying the vector has undefined behaviour.
No object of type T has started its lifetime at the memory location. This is super bad when T is not trivial.
The reinterpretation violates the strict aliasing rules.
First is fixed by value-initialising the storage. Or by making the vector non-copyable and non-movable.
Second is fixed by using placement new.
Third is technically fixed by using using the pointer returned by placement new, but you can avoid storing that pointer by std::laundering after reinterpreting the storage.
I am trying to create a static container which has stack based memory and can hold N instances of T. Much alike std::vector I want currently unused memory to not contain initialized items of T. This is usually solved with placement new but that's not possible to use in constexpr.
Using unions
I found a trick that you can use a union for this as follows:
template <typename value_type>
union container_storage_type
{
struct empty{};
constexpr container_storage_type(): uninitialized{}{}
constexpr container_storage_type(value_type v): value(v){}
constexpr void set(value_type v)
{
*this = literal_container_storage_type{v};
}
empty uninitialized;
value_type value;
};
This lets you store items uninitialized by setting the empty member and this works around the limitation that all members in constexpr have to be initialized.
Now the problem with this approach is that if value_typeis a type that implements operator=, the rule for unions says:
If a union contains a non-static data member with a non-trivial special member function (copy/move constructor, copy/move assignment, or destructor), that function is deleted by default in the union and needs to be defined explicitly by the programmer.
This means that to be able to use this trick, I need to implement operator= in the union too, but how would that look?
constexpr container_storage_type& operator=(const container_storage_type& other)
{
value = other.value; //ATTEMPT #1
//*this = container_storage_type(other.value);ATTEMPT #2
return *this;
}
Attempt #1: This does not seem possible as the compiler complains that changing the active member of a union is simply disallowed in constant expressions.
Attempt #2: This works in the set() method from the previous snippet, as it doesn't change the active member per se, but reassigns the whole union. This trick seems unable to be used in the assignment operator however since that causes endless recursion...
Am I missing something here, or is this truly a dead end for using unions as a placement-new alternative in constexpr?
Are there other alternatives to placement new that I have completely missed?
https://godbolt.org/z/km0nTY Code that illustrates the problem
In C++17, you can't.
The current restrictions on what you cannot do in constant expressions include:
an assignment expression ([expr.ass]) or invocation of an assignment operator ([class.copy.assign]) that would change the active member of a union;
a new-expression;
There really is no way around that.
In C++20, you will be able to, but probably not the way you think. The latter restriction is going to be relaxed in C++20 as a result of P0784 to something like:
a new-expression (8.3.4), unless the selected allocation function is a replaceable global allocation function (21.6.2.1, 21.6.2.2);
That is, new T will become fine but new (ptr) T will still not be allowed. As part of making std::vector constexpr-friendly, we need to be able to manage "raw" memory - but we still can't actually manage truly raw memory. Everything still has to be typed. Dealing with raw bytes is not going to work.
But std::allocator doesn't entirely deal in raw bytes. allocate(n) gives you a T* and construct takes a T* as a location and a bunch of arguments and creates a new object at that location. You may be wondering at this point how this is any different from placement new - and the only difference is that sticking with std::allocator, we stay in the land of T* - but placement new uses void*. That distinction turns out to be critical.
Unfortunately, this has the interesting consequence of your constexpr version "allocates" memory (but it allocates compiler memory, which will get elevated to static storage as necessary - so this does what you want) - but your pure runtime version surely does not want to allocate memory, indeed the whole point would be that it does not. To that end, you will have to use is_constant_evaluated() to switch between the allocating at constant evaluation time and non-allocating at runtime. This is admittedly not beautiful, but it should work.
Your storage can look something like this:
// For trivial objects
using data_t = const array<remove_const_t<T>, Capacity>>;
alignas(alignof(T)) data_t data_{};
// For non-trivial objects
alignas(alignof(T)) aligned_storage_t<T> data_[Capacity]{};
This will allow you to create a const array of non-const objects. Then constructing objects will look something like this:
// Not real code, for trivial objects
data_[idx] = T(forward<Args>(args)...);
// For non-trivial objects
new (end()) T(forward<Args>(args)...);
Placement new is mandatory here. You will be able to have the storage at compile-time, but you cannot construct it at compile-time for non-trivial objects.
You will also need to take into account whether or not your container is zero-sized, etc. I suggest you look at existing implementations for fixed sized vectors and there are even some proposals for constexpr fixed sized vectors like p0843r1.
I have done a very simple implementation of vector. I admit that I have cheated a lot and the only instance variables are for example size_t sz and T * elem. The one of many problems (but this one is the only problem I know of which may cause a crash) is the problem with pop_back().
template <typename T>
void vec<T>::pop_back() {
if (sz == 0)
return;
elem[sz-1].~T();
--sz;
}
The vector elem is a dynamically allocated array, but the objects in the array may or may not be. The problem I can see here is that this may crash in case I destroy an object in this array. This can happen in case I would destroy elements multiple times. So to say, in case I have used pop_back. This does not seem to happen for my compiler (which seems strange), but I have heard about someone having this problem with this function. I have been trying to do some research on the web and found another alternative. The zero check is not done here (probably laziness), and this example is using reinterpret_cast
template<class T>
void Vector<T>::pop_back() {
// You way want to do a check for empty() before calling the destructor.
// Call the destructor.
(reinterpret_cast<T*>(buffer)[_size-1]).~T();
// It is usual to prefer the pre decrement (and pre increment).
--_size;
}
The question is, Can I really use reinterpret_cast to be able to destroy non dynamically allocated objects in my vector class?
EDIT
By request I will show my push_back, which I was unsure how to properly write with the short time span I could spend on this. I admit that the main purpose of the class was not efficiency but a good way to handle resources instead of raw dynamic arrays (I can of course use a vector but this is rather a question about a smaller scope than general strategy so I would appreciate if the use-std::vector discussion was left out)
template <typename T>
void vec<T>::push_back(const T& obj) {
T* tmp = new T[sz+1];
if (sz > 0){
uninitialized_copy(elem, elem+sz, tmp);
delete[] elem;
}
elem = tmp;
elem[sz] = T{obj};
++sz;
}
The main problem was the capacity. I realized the capacity part would require a lot of work and without that I could just create new elements.
This answer is inspired by the stack implementation in the book Exceptional C++ by Herb Sutter (Item 12):
You can allocate memory for your vec, if you initialize your T * elem variable with
elem = static_cast<T*>(sz == 0 ? nullptr : operator new(sizeof(T)*sz));
This will give you memory on which no object has yet been constructed.
Edit(thanks to Cheers and hth. - Alf): You can also use the allocator instead of the above code by calling a.allocate(sz).
If you want to add new elements to the vector, you can use construct [1]. To call the destructor of an object you can use destroy [2].
If you now keep track on how many elements there are allocated in your vector in a variable used, you can deallocate the last by calling destroy(elem+used);
construct and destroy use the placement new and an explicit destructor call inside them. I would recommend using these functions instead of raw placement news and destructor calls.