Is it well defined behavoir to rely on a heap which zero initialises memory to initialise members to zero.
Looking at a basic POD struct:
struct S {int i;};
If I have (through whatever mechanism) a heap implementation which guarantees to zero out the backing memory, can I avoid zeroing out the members of my POD?
Another way to put this - is the following well defined behaviour:
#include <cstdlib>
#include <cstring>
#include <memory>
#include <assert.h>
struct S {int i;};
int main()
{
void* backing_mem = std::aligned_alloc(alignof(S), sizeof(S));
std::memset(backing_mem, 0, sizeof(S));
S* s = new (backing_mem)S;
assert(s->i == 0);
// or assuming we've overridden the global allocator to one which zeros memory:
S* s2 = new S;
assert(s2->i == 0);
}
Link: https://godbolt.org/z/M6oY9q5xW
I understand the language has exceptions for global/statically allocated objects to be implicitly zero, and was wondering if the same happens here. Obviously this wont work for non-PODs. Please quote the standard if possible.
(Please ignore talk about optimisation here, this is a language/correctness question)
Is it well defined behavoir to rely on a heap which zero initialises memory to initialise members to zero.
From a standard perspective: no.
Whilst you're fine from a lifetime perspective ([basic.life]/6) the s->i is an uninitialized read which is, as per [basic.indet]/2, undefined behaviour.
[basic.indet]/1 When storage for an object with automatic or dynamic storage duration is obtained, the object has an indeterminate value, and if no initialization is performed for the object, that object retains an indeterminate value until that value is replaced ([expr.ass]).
[basic.indet]/2 If an indeterminate value is produced by an evaluation, the behavior is undefined except in the following cases:
[... none applies here]
Related
I'm wondering if the following is undefined?
int main()
{
struct Doggy { int a; ~Doggy() {} };
Doggy* p = new Doggy[100];
p[50].~Doggy();
p[50].a = 3; // Is this not allowed? The destructor was called on an
// object occupying that area of memory.
// Can I access it safely?
if (p[50].a == 3);
}
I guess this is generally good to know, but the reason I'm specifically wanting to know is that I have a data structure consisting of an array, where the buckets can be nullable by setting a value, kind of like buckets in a hash table array. And when the bucket is emptied the destructor is called, but then checking and setting the null state after the destructor is called I'm wondering if it's illegal.
To elaborate a little, say I have an array of objects and each object can be made to represent null in each bucket, such as:
struct Handle
{
int value = 0; // Zero is null value
~Handle(){}
};
int main()
{
Handle* p = new Handle[100];
// Remove object 50
p[50].~Handle();
p[50].value = 0; // Set to null
if (p[50].value == 0) ; // Then it's null, can I count on this?
// Is this defined? I'm accessing memory that was occupied by
// object that was destroyed.
}
Yes it'll be UB:
[class.dtor/19]
Once a destructor is invoked for an object, the object's lifetime ends; the behavior is undefined if the destructor is invoked for an object whose lifetime has ended ([basic.life]).
[Example 2: If the destructor for an object with automatic storage duration is explicitly invoked, and the block is subsequently left in a manner that would ordinarily invoke implicit destruction of the object, the behavior is undefined. — end example]
p[50].~Handle(); and later delete[] p; will make it call the destructor for an object whose lifetime has ended.
For p[50].value = 0; after the lifetime of the object has ended, this applies:
[basic.life/6]
Before the lifetime of an object has started but after the storage which the object will occupy has been allocated or, after the lifetime of an object has ended and before the storage which the object occupied is reused or released, any pointer that represents the address of the storage location where the object will be or was located may be used but only in limited ways. For an object under construction or destruction, see [class.cdtor]. Otherwise, such a pointer refers to allocated storage ([basic.stc.dynamic.allocation]), and using the pointer as if the pointer were of type void* is well-defined. Indirection through such a pointer is permitted but the resulting lvalue may only be used in limited ways, as described below. The program has undefined behavior if:
6.2 - the pointer is used to access a non-static data member or call a non-static member function of the object
Yes, it's mostly. Handle::value is just an offset to a pointer of type Handle, so it's just going to work wherever you point it to, even if the containing object isn't currently constructed. If you were to use anything with virtual keyword, this would end up broken though.
p[50].~Handle(); this however is a different beast. You should never invoke destructors manually unless you have also explicitly invoked the constructor with placement new. Still not illegal, but dangerous.
delete[] p; (omitted in your example!) is where you end up with double-destruction, at which point you are well beyond UB, straight up in the "it's broken" domain.
Code:
#include <cstdio>
#include <new>
struct Foo {
char ch;
~Foo() { ++ch; }
};
int main() {
static_assert(sizeof(Foo) == 1);
char buffer;
auto const* pc = new (&buffer) Foo{42};
// Change value using only the const pointer
std::printf("%d\n", +buffer);
pc->~Foo();
std::printf("%d\n", +buffer);
}
godbolt
I am not causing any UB as far as I can tell, but GCC and Clang disagree on the result. I think the output should obviously be "42 43". That is the case for Clang, but GCC thinks the output is "42 0". How is that possible? Who zeros out the buffer? Am I missing something?
Your code has undefined behavior. The storage for buffer has been reused for the Foo object you created, so it's lifetime has ended and you can no longer use it. The relevent section of the standard is [basic.life]/1 with 1.5 being the relavent sub section.
The lifetime of an object o of type T ends when: [...]
the storage which the object occupies is released, or is reused by an object that is not nested within o ([intro.object]).
In your final line, the lvalue buffer doesn't access any object.
The char object that was there initially had its lifetime ended by reusing its storage for a Foo. The Foo had its lifetime ended by invoking the destructor. And no one created any object in the storage after that.
lvalue-to-rvalue conversion (which is what +buffer does, but passing buffer as an argument to a variadic function would too) is not permitted where no object exists.
§6.7.3.5
A program may end the lifetime of any object by reusing the
storage which the object occupies ...[cut]
You are accessing buffer after its lifetime is expired.
https://en.wikipedia.org/wiki/Resource_acquisition_is_initialization
There is an example of how RAII works. I always thought that C++ gets this behavior from C. That when you define a variable in a function, that variable becomes invalid when you leave the function. Though maybe the concept has no meaning when there is no object. C does not initialize structs but C++ does. Is that the difference? I am a bit confused.
I always thought that ... That when you define a variable in a function, that variable becomes invalid when you leave the function
You've thought correctly.
You seem to be confused about what RAII is for. It is for management of dynamic resources such as dynamic memory allocations. It relies on language features such a constructors and destructors, which do not exist in C.
The difference is that C++ has constructors and destructors.
C doesn't guarantee anything will be done on scope entrance and exit. If you declare a variable and don't assign anything to it, you can read garbage when you try to read that variable. When you exit a scope, nothing is done to what was on the stack in the scope you just exited.
In C++, trivial types like int behave the same way. With class types (classes, struct, unions), a variable of the type is created with a constructor and destroyed with a destructor. If you declare a variable in a way that calls a non-trivial constructor, then that constructor performs initialization on the variable. If you declare a scoped variable of a type has a non-trivial destructor, that destructor is run on scope exit to clean up the variable. Use of this construction and destruction mechanism is what is usually meant by RAII in C++.
In C, this programming error can easily happen.
typedef struct
{
int *data;
} Trivial_C;
void
my_c_function(void)
{
Trivial_C t;
t.data=malloc(5*sizeof(int));
... do something with t.data ...
} // oops! t does not exist anymore but the allocated
// memory that was known through t.data still exists!
In C++, RAII relies on destructors to do some cleanup
when an object disappears.
struct Trivial_Cpp
{
int *data;
Trivial_Cpp() : data{new int[5]} {} // data is allocated at creation
~Trivial_Cpp() { delete[] data; } // data is released at destruction
};
void
my_cpp_function()
{
Trivial_Cpp t;
... do something with t.data ...
} // OK, t does not exist anymore and the destructor has
// been called, so the allocated memory has been released
Of course these code snippets are trivial and largely incomplete.
Moreover, you rarely need to allocate memory by yourself; std::vector
for example will do it perfectly for you (because it uses RAII).
I always thought that C++ gets this behavior from C.
No, C++ and C are wildly different languages (by now) and differ in many aspects, often surprising.
First, you should understand different ways of lifetimes of objects (an object, in C speak, is a thing that uses memory, it hasn't got anything to do with Object Oriented Programming). In C there are three kinds of lifetimes:
static: Things that get allocated before the program start and destroyed afterwards. All global variables or static local variables. Static objects are implicitly initialized to 0/0.0 or NULL for pointer objects respectively.
dynamic: Objects that are created with the help of an allocation function such as malloc which in turn usually calls the OS via syscall (eg. sys_brk) to create space on the heap. This is an implementation detail though, C is very vague about how exactly the dynamic memory is acquired and has no notion of the heap.
When you don't need the memory anymore, you should free the object.
automatic: This is what happens when you simply create local variable without much thinking. The space is usually allocated on the stack and thus it will be invalid as soon as it leaves its scope.
const int GLOBAL_VARIABLE; // static lifetime
struct s {
int a;
}
void foo(void)
{
static int local_static_variable; // static lifetime as well.
int x; // automatic lifetime
struct s t; // automatic as well
struct s *p = malloc(sizeof (*p)); // while the pointer p itself is automatic, it points to an object of dynamic lifetime
{
int y; // automatic variable
}
// y out of scope: "freed"
free(p); // object pointed to by p must be freed.
} // x and p go out of scope
C++ with RAII introduces the usability of automatic freeing when something goes out of scope for more complex structures and objects allocated on the heap. For example you may have a structure that contains pointers objects which contain pointers to objects etc. In C, automatic allocation of the first-level structure only allocates memory for the pointers but not the objects within. Nor, if you allocate the memory for those objects using malloc, will it free them if the primary structure goes out of scope:
struct s {
int *array;
}
void foo(void)
{
struct s; // structure automatically allocated, but s.array is undefined
s.array = calloc(10, sizeof (*s.array)); // allocate space for 10 elements in array
s.array[0] = 0xf00;
free(s.array); // must free space acquired earlier
}
// s goes out of scope.
// if we hadn't free'd s.array, the allocated space would still "be there"
C++ allows you to define constructors and destructors for each object and subobject that will automatically allocate and free all subobjects, s.t. if s would go out of scope, s.array would be deallocated as well.
I am struggling with implementing a shared memory buffer without breaking C99's strict aliasing rules.
Suppose I have some code that processes some data and needs to have some 'scratch' memory to operate. I could write it as something like:
void foo(... some arguments here ...) {
int* scratchMem = new int[1000]; // Allocate.
// Do stuff...
delete[] scratchMem; // Free.
}
Then I have another function that does some other stuff that also needs a scratch buffer:
void bar(...arguments...) {
float* scratchMem = new float[1000]; // Allocate.
// Do other stuff...
delete[] scratchMem; // Free.
}
The problem is that foo() and bar() may be called many times during operation and having heap allocations all over the place may be quite bad in terms of performance and memory fragmentation. An obvious solution would be to allocate a common, shared memory buffer of proper size once and then pass it into foo() and bar() as an argument, BYOB-style:
void foo(void* scratchMem);
void bar(void* scratchMem);
int main() {
const int iAmBigEnough = 5000;
int* scratchMem = new int[iAmBigEnough];
foo(scratchMem);
bar(scratchMem);
delete[] scratchMem;
return 0;
}
void foo(void* scratchMem) {
int* smem = (int*)scratchMem;
// Dereferencing smem will break strict-aliasing rules!
// ...
}
void bar(void* scratchMem) {
float* smem = (float*)scratchMem;
// Dereferencing smem will break strict-aliasing rules!
// ...
}
I guess I have two questions now:
- How can I implement a shared common scratch memory buffer that is not in violation of aliasing rules?
- Even though the above code does violate strict aliasing rules, there is no 'harm' being done with the alias. Therefore could any sane compiler generate (optimized) code that still gets me into trouble?
Thanks
Actually, what you have written is not a strict aliasing violation.
C++11 spec 3.10.10 says:
If a program attempts to access the stored value of an object through a glvalue of other than one of the
following types the behavior is undefined
So the thing that causes the undefined behavior is accessing the stored value, not just creating a pointer to it. Your example does not violate anything. It would need to do the next step: float badValue = smem[0]. smem[0] gets the stored value from the shared buffer, creating an aliasing violation.
Of course, you aren't about to just grab smem[0] before setting it. You are going to write to it first. Assigning to the same memory does not access the stored value, so no ailiasing However, it IS illegal to write over the top of an object while it is still alive. To prove that we are safe, we need object lifespans from 3.8.4:
A program may end the lifetime of any object by reusing the storage which the object occupies or by
explicitly calling the destructor for an object of a class type with a non-trivial destructor. For an object
of a class type with a non-trivial destructor, the program is not required to call the destructor explicitly
before the storage which the object occupies is reused or released; ... [continues on regarding consequences of not calling destructors]
You have a POD type, so trivial destructor, so you can simply declare verbally "the int objects are all at the end of their lifespan, I'm using the space for floats." You then reuse the space for floats, and no aliasing violation occurs.
It is always valid to interpret an object as a sequence of bytes (i.e. it is not an aliasing violation to treat any object pointer as the pointer to the first element of an array of chars), and you can construct an object in any piece of memory that's large enough and suitably aligned.
So, you can allocate a large array of chars (any signedness), and locate an offset that's aliged at alignof(maxalign_t); now you can interpret that pointer as an object pointer once you've constructed the appropriate object there (e.g. using placement-new in C++).
You do of course have to make sure not to write into an existing object's memory; in fact, object lifetime is intimately tied to what happens to the memory which represents the object.
Example:
char buf[50000];
int main()
{
uintptr_t n = reinterpret_cast<uintptr_t>(buf);
uintptr_t e = reinterpret_cast<uintptr_t>(buf + sizeof buf);
while (n % alignof(maxalign_t) != 0) { ++n; }
assert(e > n + sizeof(T));
T * p = :: new (reinterpret_cast<void*>(n)) T(1, false, 'x');
// ...
p->~T();
}
Note that memory obtained by malloc or new char[N] is always aligned for maximal alignment (but not more, and you may wish to use over-aligned addresses).
If a union is used to hold the int and float variables, then you can by pass the strict aliasing. More about this is given in
http://cellperformance.beyond3d.com/articles/2006/06/understanding-strict-aliasing.html
Also see the following article.
http://blog.regehr.org/archives/959
He gives a way to use unions to do this.
Take a small example where, I am trying to find out if a variable is allocated on heap or not:
struct A
{
bool isOnHeap;
A () {} // not touching isOnHeap
~A () {}
void* operator new (size_t size)
{
A* p = (A*) malloc(size);
p->isOnHeap = true; // setting it to true
return p;
}
void operator delete (void *p) { free(p); }
};
It gives expected result in g++-4.5 (with warning for stack object). Is it ill defined
to do such operations ?
You can't initialize class members in an overloaded operator new because the object's lifetime hasn't started. You can only initialize members during the construction of the object.
You have no guarantee that the implementation won't wipe the memory between the time operator new returns and the time the object's construction starts or that during object construction members that are specified to have an indeterminate value by the standard (e.g. because they are POD and not explicitly initialized in the constructor like isOnHeap) aren't deliberately set to something by the implementation.
Note that A has a non-trivial constructor (it is user-declared), so its lifetime doesn't start when the storage for the object is allocated (ISO/IEC 14882:2003, 3.8 [basic.life] / 1) and the program has undefined behavior if it uses a pointer to the storage for the object to access a non-static data member (3.8 / 5). Even if A was a POD type, it's value after the completion of the new-expression would still be indeterminate rather than necessarily being related to the values in the bytes in the storage for the object before the new-expression was evaluated.
As Charles said, the object only comes to lifetime after it has been newed, so setting data within your implementation of new is rather dangerous.
Also, when your developers use tools like Lint, there's a big chance that it complains that the member isOnHeap is not initialized in the constructor. If then someone thinks "hey, Lint is right, let's initialize isOnHeap in the constructor of A", this will undermine the mechanism that you try to achieve.
There is a second case of which you probably didn't think. Suppose that someone writes this:
class MyClass
{
public:
...
private:
struct A m_a;
};
int main()
{
MyClass *myVariable = new MyClass();
}
Then your implementation of new will not be called. Nevertheless the instance of A is allocated on the heap (as part of the MyClass instance).
Can you explain why you want to know whether something has been allocated on the heap or not? Maybe there's another, more elegant solution to your problem.
Even when not considering the operator new itself (which is nonstandard and I would even say ugly, but knowing the exact details of some particular compiler it might be workable), there is another problem with this, which renders it useless anyway: You have no guarantee the value od isOnHeap will not be true when allocated on the stack. The stack is not initialized and any garbage from function invocations done before can be found there.