Currently, I am reading the source code of Protocol Buffer, and I found one weird enum codes defined here
~scoped_ptr() {
enum { type_must_be_complete = sizeof(C) };
delete ptr_;
}
void reset(C* p = NULL) {
if (p != ptr_) {
enum { type_must_be_complete = sizeof(C) };
delete ptr_;
ptr_ = p;
}
}
Why the enum { type_must_be_complete = sizeof(C) }; is defined here? what is it used for?
This trick avoids UB by ensuring that definition of C is available when this destructor is compiled. Otherwise the compilation would fail as the sizeof incomplete type (forward declared types) can not be determined but the pointers can be used.
In compiled binary, this code would be optimized out and would have no effect.
Note that: Deleting incomplete type may be undefined behavior from 5.3.5/5:.
if the object being deleted has incomplete class type at the point of
deletion and the complete class has a non-trivial destructor or a
deallocation function, the behavior is undefined.
g++ even issues the following warning:
warning: possible problem detected in invocation of delete
operator: warning: 'p' has incomplete type warning: forward
declaration of 'struct C'
sizeof(C) will fail at compile time if C is not a complete type. Setting a local scope enum to it makes the statement benign at runtime.
It's a way of the programmer protecting themselves from themselves: the behaviour of a subsequent delete ptr_ on an incomplete type is undefined if it has a non-trivial destructor.
To understand the enum, start with considering the destructor without it:
~scoped_ptr() {
delete ptr_;
}
where ptr_ is a C*. If type C is incomplete at this point, i.e. all that the compiler knows is struct C;, then (1)a default-generated do-nothing destructor is used for the C instance pointed to. That's unlikely to be the right thing to do for an object managed by a smart pointer.
If deleting via a pointer to incomplete type had always had Undefined Behavior, then the standard could just require the compiler to diagnose it and fail. But it's well-defined when the real destructor is trivial: knowledge that the programmer can have, but the compiler doesn't have. Why the language defines and allows this is beyond me, but C++ supports many practices that today are not regarded as best practices.
A complete type has a known size, and hence, sizeof(C) will compile if and only if C is a complete type -- with known destructor. So it can be used as a guard. One way would be simply
(void) sizeof(C); // Type must be complete
I would guess that with some compiler and options the compiler optimizes it away before it could notice that it shouldn't compile, and that the enum is a way to avoid such non-conforming compiler behavior:
enum { type_must_be_complete = sizeof(C) };
An alternative explanation for the choice of enum rather than just a discarded expression, is simply personal preference.
Or as James T. Hugget suggests in a comment to this answer, “The enum may be a way of creating a pseudo-portable error message at compile time”.
(1) The default-generated do-nothing destructor for an incomplete type was a problem with old std::auto_ptr. It was so insidious that it made its way into a GOTW item about the PIMPL idiom, written by the chair of the international C++ standardization committee Herb Sutter. Of course, nowadays that std::auto_ptr is deprecated, one will instead use some other mechanism.
Maybe a trick to be sure C is defined.
Related
Have a look at is simple example:
struct Base { /* some virtual functions here */ };
struct A: Base { /* members, overridden virtual functions */ };
struct B: Base { /* members, overridden virtual functions */ };
void fn() {
A a;
Base *base = &a;
B *b = reinterpret_cast<B *>(base);
Base *x = b;
// use x here, call virtual functions on it
}
Does this little snippet have Undefined Behavior?
The reinterpret_cast is well defined, it returns an unchanged value of base, just with the type of B *.
But I'm not sure about the Base *x = b; line. It uses b, which has a type of B *, but it actually points to an A object. And I'm not sure, whether x is a "proper" Base pointer, whether virtual functions can be called with it.
static_cast (or an implicit derived-to-base-pointer conversion, which does exactly the same thing) is substantially different from reinterpret_cast. There is no guarantee that that the base subobject starts at the same address as the complete object.
Most implementations place the first base subobject at the same address as the complete object, but of course even such implementations cannot place two different non-empty base subobjects at the same address. (An object with virtual functions is not empty). When the base subobject is not at the same address as the complete object, static_cast is not a no-op, it involves pointer adjustment.
There are implementations that never place even the first base subobject at the same address as the complete object. It is allowed to place the base subobject after all members of derived, for example. IIRC the Sun C++ compiler used to layout classes this way (don't know if it's still doing that). On such an implementation, this code is almost guaranteed to fail.
Similar code with B having more than one base will fail on many implementations. Example.
The reinterpret_cast is valid (the result can be dereferenced) if the two classes are layout-compatible; that is
they both have standard layout,
they both have the same non-static data members
But the classes do not have standard layout because one of the requirements of StandardLayoutType it that the class has no virtual functions or virtual base classes.
Regarding the validity of pointers derived from conversions, the standard has this to say in the section on "Safely-derived pointers":
6.7.4.3 Safely-derived pointers
4. An implementation may have relaxed pointer safety, in which case the validity of a pointer value does not depend on whether it is a safely-derived pointer value. Alternatively, an implementation may have strict pointer safety, in which case a pointer value referring to an object with dynamic storage duration that is not a safely-derived pointer value is an invalid pointer value unless the referenced complete object has previously been declared reachable. [ Note: The effect of using an invalid pointer value (including passing it to a deallocation function) is undefined, see 6.7.4.2. This is true even if the unsafely-derived pointer value might compare equal to some safely-derived pointer value. —end note ] It is implementation-defined whether an implementation has relaxed or strict pointer safety.
Yes, It does have undefined behavior. The layout about suboject of Base in A and B is undefined. x may be not a real Base oject.
If A and B are a verbatim copy of each other (except for their names) and are declared in the same context (same namespace, same #defines, no __LINE__ usage), then common C++ compilers (gcc, clang) will produce two binary representations which are fully interchangeable.
If A and B use the same method signatures but the bodies of corresponding methods differ, it is unsafe to cast A* to B* because the optimization pass in the compiler could for example partially inline the body of void B::method() at the call site b->method() while the programmer's assumption could be that b->method() will call A::method(). Therefore, as soon as the programmer uses an optimizing compiler the behavior of accessing A through type B* becomes undefined.
Problem: All compilers are always at least to some extent "optimizing" the source code passed to them, even at -O0. In cases of behavior not mandated by the C++ standard (that is: undefined behavior), the compiler's implicit assumptions - when all optimizations are turned off - might differ from programmer's assumptions. The implicit assumptions have been made by the developers of the compiler.
Conclusion: If the programmer is able to avoid using an optimizing compiler then it is safe to access A via B*. The only issue such a programmer needs to tackle with is that non-optimizing compilers do not exist.
A managed C++ implementation might abort the program when A* is casted to B* via reinterpret_cast, when b->field is accessed, or when b->method() is called. Some other managed C++ implementation might try harder to avoid a program crash and so it will resort to temporary duck typing when it sees the program accessing A via B*.
Some questions are:
Can the programmer guess what the managed C++ implementation will do in cases of behavior not mandated by the C++ standard?
What if the programmer sends the code to another programmer who will pass it to a different managed C++ implementation?
If a case isn't covered by the C++ standard, does it mean that a C++ implementation can choose to do anything it considers appropriate in order to cope with the case?
Regarding the following code:
class One {
public:
double number{};
};
class Two {
public:
int integer{};
}
class Mixture {
public:
double& foo() {
new (&storage) One{1.0};
return reinterpret_cast<One*>(&storage)->number;
}
int& bar() {
new (&storage) Two{2};
return reinterpret_cast<Two*>(&storage)->integer;
}
std::aligned_storage_t<8> storage;
};
int main() {
auto mixture = Mixture{};
cout << mixture.foo() << endl;
cout << mixture.bar() << endl;
}
I haven't called the destructor for the types because they are trivially destructible. My understanding of the standard is that for this to be safe, we would need to launder the pointer to storage before passing it to the reinterpret_cast. However, std::optional's implementation in libstdc++ does not seem to use std::launder() and simply constructs the object right into the union storage. https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/include/std/optional.
Is my example above well defined behavior? What do I need to do to make it work? Would a union make this work?
In your code, you do need std::launder in order to make your reinterpret_cast do what you want it to do. This is a separate issue from that of re-using memory. According to the standard ([expr.reinterpret].cast]7), your expression
reinterpret_cast<One*>(&storage)
is equivalent to:
static_cast<One*>(static_cast<void*>(&storage))
However, the outer static_cast does not succeed in producing a pointer to the newly created One object because according to [expr.static.cast]/13,
if the original pointer value points to an object a, and there is an object b of type T (ignoring cv-qualification) that is pointer-interconvertible (6.9.2)
with a, the result is a pointer to b. Otherwise, the pointer value is unchanged by the conversion.
That is, the resulting pointer still points to the storage object, not to the One object nested within it, and using it as a pointer to a One object would violate the strict aliasing rule. You must use std::launder to force the resulting pointer to point to the One object. Or, as pointed out in the comments, you could simply use the pointer returned by placement new directly, rather than the one obtained from reinterpret_cast.
If, as suggested in the comments, you used a union instead of aligned_storage,
union {
One one;
Two two;
};
you would sidestep the pointer-interconvertibility issue, so std::launder would not be needed on account of non-pointer-interconvertibility. However, there is still the issue of re-use of memory. In this particular case, std::launder is not needed on account of re-use because your One and Two classes do not contain any non-static data members of const-qualified or reference type ([basic.life]/8).
Finally, there was the question of why libstdc++'s implementation of std::optional does not use std::launder, even though std::optional may contain classes that contain non-static data members of const-qualified or reference type. As pointed out in comments, libstdc++ is part of the implementation, and may simply elide std::launder when the implementers know that GCC will still compile the code properly without it. The discussion that led up to the introduction of std::launder (see CWG 1776 and the linked thread, N4303, P0137) seems to indicate that, in the opinion of people who understand the standard much better than I do, std::launder is indeed required in order to make the union-based implementation of std::optional well-defined in the presence of members of const-qualified or reference type. However, I am not sure that the standard text is clear enough to make this obvious, and it might be worth having a discussion about how it might be clarified.
We just upgraded our compiler to gcc 4.6 and now we get some of these warnings. At the moment our codebase is not in a state to be compiled with c++0x and anyway, we don't want to run this in prod (at least not yet) - so I needed a fix to remove this warning.
The warnings occur typically because of something like this:
struct SomeDataPage
{
// members
char vData[SOME_SIZE];
};
later, this is used in the following way
SomeDataPage page;
new(page.vData) SomeType(); // non-trivial constructor
To read, update and return for example, the following cast used to happen
reinterpret_cast<SomeType*>(page.vData)->some_member();
This was okay with 4.4; in 4.6 the above generates:
warning: type punned pointer will break strict-aliasing rules
Now a clean way to remove this error is to use a union, however like I said, we can't use c++0x (and hence unrestricted unions), so I've employed the horrible hack below - now the warning has gone away, but am I likely to invoke nasal daemons?
static_cast<SomeType*>(reinterpret_cast<void*>(page.vData))->some_member();
This appears to work okay (see simple example here: http://www.ideone.com/9p3MS) and generates no warnings, is this okay(not in the stylistic sense) to use this till c++0x?
NOTE: I don't want to use -fno-strict-aliasing generally...
EDIT: It seems I was mistaken, the same warning is there on 4.4, I guess we only picked this up recently with the change (it was always unlikely to be a compiler issue), the question still stands though.
EDIT: further investigation yielded some interesting information, it seems that doing the cast and calling the member function in one line is what is causing the warning, if the code is split into two lines as follows
SomeType* ptr = reinterpret_cast<SomeType*>(page.vData);
ptr->some_method();
this actually does not generate a warning. As a result, my simple example on ideone is flawed and more importantly my hack above does not fix the warning, the only way to fix it is to split the function call from the cast - then the cast can be left as a reinterpret_cast.
SomeDataPage page;
new(page.vData) SomeType(); // non-trivial constructor
reinterpret_cast<SomeType*>(page.vData)->some_member();
This was okay with 4.4; in 4.6 the above generates:
warning: type punned pointer will break strict-aliasing rules
You can try:
SomeDataPage page;
SomeType *data = new(page.vData) SomeType(); // non-trivial constructor
data->some_member();
Why not use:
SomeType *item = new (page.vData) SomeType();
and then:
item->some_member ();
I don't think a union is the best way, it may also be problematic. From the gcc docs:
`-fstrict-aliasing'
Allows the compiler to assume the strictest aliasing rules
applicable to the language being compiled. For C (and C++), this
activates optimizations based on the type of expressions. In
particular, an object of one type is assumed never to reside at
the same address as an object of a different type, unless the
types are almost the same. For example, an `unsigned int' can
alias an `int', but not a `void*' or a `double'. A character type
may alias any other type.
Pay special attention to code like this:
union a_union {
int i;
double d;
};
int f() {
a_union t;
t.d = 3.0;
return t.i;
}
The practice of reading from a different union member than the one
most recently written to (called "type-punning") is common. Even
with `-fstrict-aliasing', type-punning is allowed, provided the
memory is accessed through the union type. So, the code above
will work as expected. However, this code might not:
int f() {
a_union t;
int* ip;
t.d = 3.0;
ip = &t.i;
return *ip;
}
How this relates to your problem is tricky to determine. I guess the compiler is not seeing the data in SomeType as the same as the data in vData.
I'd be more concerned about SOME_SIZE not being big enough, frankly. However, it is legal to alias any type with a char*. So simply doing reinterpret_cast<T*>(&page.vData[0]) should be just fine.
Also, I'd question this kind of design. Unless you're implementing boost::variant or something similar, there's not much reason to use it.
struct SomeDataPage
{
// members
char vData[SOME_SIZE];
};
This is problematic for aliasing/alignment reasons. For one, the alignment of this struct isn't necessarily the same as the type you're trying to pun inside it. You could try using GCC attributes to enforce a certain alignment:
struct SomeDataPage { char vData[SOME_SIZE] __attribute__((aligned(16))); };
Where alignment of 16 should be adequate for anything I've come across. Then again, the compiler still won't like your code, but it won't break if the alignment is good. Alternatively, you could use the new C++0x alignof/alignas.
template <class T>
struct DataPage {
alignof(T) char vData[sizeof(T)];
};
In one of the projects I'm working on, I'm seeing this code
struct Base {
virtual ~Base() { }
};
struct ClassX {
bool isHoldingDerivedObj() const {
return typeid(1 ? *m_basePtr : *m_basePtr) == typeid(Derived);
}
Base *m_basePtr;
};
I have never seen typeid used like that. Why does it do that weird dance with ?:, instead of just doing typeid(*m_basePtr)? Could there be any reason? Base is a polymorphic class (with a virtual destructor).
EDIT: At another place of this code, I'm seeing this and it appears to be equivalently "superfluous"
template<typename T> T &nonnull(T &t) { return t; }
struct ClassY {
bool isHoldingDerivedObj() const {
return typeid(nonnull(*m_basePtr)) == typeid(Derived);
}
Base *m_basePtr;
};
I think it is an optimisation! A little known and rarely (you could say "never") used feature of typeid is that a null dereference of the argument of typeid throws an exception instead of the usual UB.
What? Are you serious? Are you drunk?
Indeed. Yes. No.
int *p = 0;
*p; // UB
typeid (*p); // throws
Yes, this is ugly, even by the C++ standard of language ugliness.
OTOH, this does not work anywhere inside the argument of typeid, so adding any clutter will cancel this "feature":
int *p = 0;
typeid(1 ? *p : *p); // UB
typeid(identity(*p)); // UB
For the record: I am not claiming in this message that automatic checking by the compiler that a pointer is not null before doing a dereference is necessarily a crazy thing. I am only saying that doing this check when the dereference is the immediate argument of typeid, and not elsewhere, is totally crazy. (Maybe is was a prank inserted in some draft, and never removed.)
For the record: I am not claiming in the previous "For the record" that it makes sense for the compiler to insert automatic checks that a pointer is not null, and to to throw an exception (as in Java) when a null is dereferenced: in general, throwing an exception on a null dereference is absurd. This is a programming error so an exception will not help. An assertion failure is called for.
The only effect I can see is that 1 ? X : X gives you X as an rvalue instead of plain X which would be an lvalue. This can matter to typeid() for things like arrays (decaying to pointers) but I don't think it would matter if Derived is known to be a class. Perhaps it was copied from someplace where the rvalue-ness did matter? That would support the comment about "cargo cult programming"
Regarding the comment below I did a test and sure enough typeid(array) == typeid(1 ? array : array), so in a sense I'm wrong, but my misunderstanding could still match the misunderstanding that lead to the original code!
This behaviour is covered by [expr.typeid]/2 (N3936):
When typeid is applied to a glvalue expression whose type is a polymorphic class type, the result refers to a std::type_info object representing the type of the most derived object (that is, the dynamic type) to which the glvalue refers. If the glvalue expression is obtained by applying the unary * operator to a pointer and the pointer is a null pointer value, the typeid expression throws an exception of a type that would match a handler of type std::bad_typeid exception.
The expression 1 ? *p : *p is always an lvalue. This is because *p is an lvalue, and [expr.cond]/4 says that if the second and third operand to the ternary operator have the same type and value category, then the result of the operator has that type and value category also.
Therefore, 1 ? *m_basePtr : *m_basePtr is an lvalue with type Base. Since Base has a virtual destructor, it is a polymorphic class type.
Therefore, this code is indeed an example of "When typeid is applied to a glvalue expression whose type is a polymorphic class type" .
Now we can read the rest of the above quote. The glvalue expression was not "obtained by applying the unary * operator to a pointer" - it was obtained via the ternary operator. Therefore the standard does not require that an exception be thrown if m_basePtr is null.
The behaviour in the case that m_basePtr is null would be covered by the more general rules about dereferencing a null pointer (which are a bit murky in C++ actually but for practical purposes we'll assume that it causes undefined behaviour here).
Finally: why would someone write this? I think curiousguy's answer is the most plausible suggestion so far: with this construct, the compiler does not have to insert a null pointer test and code to generate an exception, so it is a micro-optimization.
Presumably the programmer is either happy enough that this will never be called with a null pointer, or happy to rely on a particular implementation's handling of null pointer dereference.
I suspect some compiler was, for the simple case of
typeid(*m_basePtr)
returning typeid(Base) always, regardless of the runtime type. But turning it to an expression/temporary/rvalue made the compiler give the RTTI.
Question is which compiler, when, etc. I think GCC had problems with typeid early on, but it is a vague memory.
I know that C++ doesn't support covariance for containers elements, as in Java or C#. So the following code probably is undefined behavior:
#include <vector>
struct A {};
struct B : A {};
std::vector<B*> test;
std::vector<A*>* foo = reinterpret_cast<std::vector<A*>*>(&test);
Not surprisingly, I received downvotes when suggesting this a solution to another question.
But what part of the C++ standard exactly tells me that this will result in undefined behavior? It's guaranteed that both std::vector<A*> and std::vector<B*> store their pointers in a continguous block of memory. It's also guaranteed that sizeof(A*) == sizeof(B*). Finally, A* a = new B is perfectly legal.
So what bad spirits in the standard did I conjure (except style)?
The rule violated here is documented in C++03 3.10/15 [basic.lval], which specifies what is referred to informally as the "strict aliasing rule"
If a program attempts to access the stored value of an object through an lvalue of other than one of the following types the behavior is undefined:
the dynamic type of the object,
a cv-qualified version of the dynamic type of the object,
a type that is the signed or unsigned type corresponding to the dynamic type of the object,
a type that is the signed or unsigned type corresponding to a cv-qualified version of the dynamic type of the object,
an aggregate or union type that includes one of the aforementioned types among its members (including, recursively, a member of a subaggregate or contained union),
a type that is a (possibly cv-qualified) base class type of the dynamic type of the object,
a char or unsigned char type.
In short, given an object, you are only allowed to access that object via an expression that has one of the types in the list. For a class-type object that has no base classes, like std::vector<T>, basically you are limited to the types named in the first, second, and last bullets.
std::vector<Base*> and std::vector<Derived*> are entirely unrelated types and you can't use an object of type std::vector<Base*> as if it were a std::vector<Derived*>. The compiler could do all sorts of things if you violate this rule, including:
perform different optimizations on one than on the other, or
lay out the internal members of one differently, or
perform optimizations assuming that a std::vector<Base*>* can never refer to the same object as a std::vector<Derived*>*
use runtime checks to ensure that you aren't violating the strict aliasing rule
[It might also do none of these things and it might "work," but there's no guarantee that it will "work" and if you change compilers or compiler versions or compilation settings, it might all stop "working." I use the scare-quotes for a reason here. :-)]
Even if you just had a Base*[N] you could not use that array as if it were a Derived*[N] (though in that case, the use would probably be safer, where "safer" means "still undefined but less likely to get you into trouble).
You are invoking the bad spirit of reinterpret_cast<>.
Unless you really know what you do (I mean not proudly and not pedantically) reinterpret_cast is one of the gates of evil.
The only safe use I know of is managing classes and structures between C++ and C functions calls.
There maybe some others however.
The general problem with covariance in containers is the following:
Let's say your cast would work and be legal (it isn't but let's assume it is for the following example):
#include <vector>
struct A {};
struct B : A { public: int Method(int x, int z); };
struct C : A { public: bool Method(char y); };
std::vector<B*> test;
std::vector<A*>* foo = reinterpret_cast<std::vector<A*>*>(&test);
foo->push_back(new C);
test[0]->Method(7, 99); // What should happen here???
So you have also reinterpret-casted a C* to a B*...
Actually I don't know how .NET and Java manage this (I think they throw an exception when trying to insert a C).
I think it'll be easier to show than tell:
struct A { int a; };
struct Stranger { int a; };
struct B: Stranger, A {};
int main(int argc, char* argv[])
{
B someObject;
B* b = &someObject;
A* correct = b;
A* incorrect = reinterpret_cast<A*>(b);
assert(correct != incorrect); // troubling, isn't it ?
return 0;
}
The (specific) issue showed here is that when doing a "proper" conversion, the compiler adds some pointer ajdustement depending on the memory layout of the objects. On a reinterpret_cast, no adjustement is performed.
I suppose you'll understand why the use of reinterpet_cast should normally be banned from the code...