I was just reading a new C++ challenge:
http://blogs.msdn.com/b/vcblog/archive/2014/02/04/challenge-vulnerable-code.aspx
The supplied code if full of issues, some obvious to anybody with good programming habits, some visible only to C++ natives :-)
It is described in the comments, that one particular line (37) is particularly dangerous:
ImageFactory::DebugPrintDimensions((ImageFactory::CustomImage*)image);
the function then calls a virtual method of CustomImage (defined first time in CustomImage).
This allegedly causes the first member of CustomImage to be treated as the vptr of the instance (it's an unique_ptr actually) and make the binary pointed by it treated as executable (perhaps malicious) code..
While I can understand this, I wonder why does this really work.
CustomImage is a virtual class, so (probably) its first 4 bytes (just assume X86) are THE vptr, and the unique_ptr member is next.. And since the cast doesn't seem to be shifting anything...
... how would it be possible to execute data held by unique_ptr?
My take (and I'm more than happy to be corrected):
Here, CustomImage is a polymorphic class (with a vptr as the first "member" under the Windows ABI), but Imageis not. The order of the definitions means that the ImageFactory functions know that CustomImage is an Image, but main() does not.
So when the factory does:
Image* ImageFactory::LoadFromDisk(const char* imageName)
{
return new (std::nothrow) CustomImage(imageName);
}
the CustomImage* pointer is converted to an Image* and everything is fine. Because Image is not polymorphic, the pointer is adjusted to point to the (POD) Image instance inside the CustomImage -- but crutially, this is after the vptr, because that always comes first in a polymorphic class in the MS ABI (I assume).
However, when we get to
ImageFactory::DebugPrintDimensions((ImageFactory::CustomImage*)image);
the compiler sees a C-style cast from one class it knows nothing about to another. All it does is take the address it has and pretend it's a CustomImage* instead. This address in fact points to the Image within the custom image; but because Image is empty, and presumably the empty base class optimisation is in effect, it ends up pointing to the first member within CustomImage, i.e. the unique_ptr.
Now, ImageFactory::DebugPrintDimensions() assumes it's been handed a pointer to a fully-complete CustomImage, so that the address is equal to the address of the vptr. But it hasn't -- it's been handed the address of the unique_ptr, because at the point at which is was called, the compiler didn't know any better. So now it dereferences what it thinks is the vptr (really data we're in control of), looks for the offset of the virtual function and blindly exectutes that -- and now we're in trouble.
There are a couple of things that could have helped mitigate this. Firstly, since we're manipulating a derived class via a base class pointer, Image should have a virtual destructor. This would have made Image polymorphic, and in all probability we wouldn't have had a problem (and we wouldn't leak memory, either).
Secondly, because we're casting from base-to-derived, dynamic_cast should have been used rather than the C style cast, which would have involved a run-time check and correct pointer adjustment.
Lastly, if the compiler had all the information to hand when compiling main(), it might have been able to warn us (or performed the cast correctly, adjusting for the polymorphic nature of CustomImage). So moving the class definitions above main() is recommended too.
Presumably, the memory layout is such that the vptr comes before the base sub-object, like this:
class CustomImage {
void * __vptr;
Image __base; // empty
unique_ptr<whatever> evil;
};
This means that a valid conversion from Image* to CustomImage* requires subtracting a few bytes from the pointer. However, the evil cast you've posted comes before the class definitions, so it doesn't know how to correctly adjust the pointer. Instead, it acts like reinterpret_cast, simply pretending that the pointer points to CustomImage without adjusting its value.
Now, since the base class is empty, the pointer inside unique_ptr will be misinterpreted as the vptr. This points to another pointer, which will be misinterpreted as the vtable's pointer to the first virtual member function. This in turn points to data loaded from the file, which will be executed as code when that virtual function is called. As the icing on the cake, the memory protection flags are loaded from the file, and not adjusted to prevent execution.
Some lessons here are:
Avoid C-style casts, especially on pointer or reference types. They fall back to reinterpret_cast, leading to a minefield of undefined behaviour, if the conversion isn't valid. (To compound the evil, their syntax is also ungreppable, and easy to miss if you don't read the code too carefully.)
Avoid non-polymorphic base classes. As well as being conceptually dubious, and making deletion more awkward and error-prone, it can do surprising things to the memory layout as we see here. If the base class were polymorphic, we could use dynamic_cast (or avoid the cast altogether by providing suitable virtual functions), with no possibility of an invalid conversion.
Avoid unnecessary levels of indirection - there's no particular need for m_imageData to be a pointer.
Never put user data in executable memory.
Related
[embedded C person here, boggled by embedded C++]
There is a class that derives from another, as follows:
class DerivedThing: public BaseThing {
...
}
There is some code, which I have no control over, which takes in a pointer to DerivedThing and, eventually, casts that pointer to BaseThing. I'm led to believe this works. Anyway, I need to implement my own DerivedThing but it needs to subclass some other stuff:
class MyDerivedThing: public BaseThing, public AnotherThing {
...
}
What do I need to do to make sure that the cast will still work? I'm asking because in my case the wrong functions are most definitely being invoked.
EDIT: The cast in the code I have no control over is:
int setInterface(void* context)
{
interface[0] = (BaseThing *) context;
...
I was overly pessimistic while commenting!
Casting to BaseThing* yourself, and passing that to the function taking void*, should work.
The problem with just passing the address of derived object, as-is, is that its address gets implicitly converted to a void*. As a result, all information about its type and the involved class hierarchy is lost.
The receiving function then assumes the now-void* pointer pointed directly at a BaseThing. This is where things start to go loopy, manifesting as symptoms like the wrong functions being called - if you're lucky - because assuming that the derived object and any particular base subobject have the same address is not (always/reliably) a valid assumption where multiple and/or virtual inheritance are at play.
Intermission: It's not immediately clear why this quoted order of bases would cause a problem:
class MyDerivedThing: public BaseThing, public AnotherThing {
...but there are various possibilities. For example, if all three classes here have no virtual methods, then there shouldn't be a problem. But, for example, if BaseThing was not virtual but either of the other two is, the compiler might put a virtual table pointer at the top of the object, which will blow up anything that just takes that address and assumes there's a BaseThing there.
Anyway - by casting yourself before passing, the compiler can do a proper type-aware cast, performing any adjusting arithmetic that might be required, to the address of the BaseThing within your object - then pass that to the function. It still goes through the conversion to void* and back, but that'll now be guaranteed to represent the address of a BaseThing in the end.
I'd still question the author on why this function takes a void*. One of the key things about C++ is type safety. void* makes a mockery of that. Worse, they just cast said void* to a BaseThing* immediately anyway. So, why not just... take a BaseThing* in the first place? Then the compiler could implicitly perform the mentioned safe typecast at any call site, rather than making you do it.
I dont see what the following macro is doing? If anyone can help me see it it would be appreciated.
#define BASE_OFFSET(ClassName,BaseName)\
(DWORD(static_cast < BaseName* >( reinterpret_cast\
< ClassName* >(Ox10000000)))-Ox10000000)
If anyone is curious to know where it is coming from, it comes out of the 3rd chapter of Don Box Book Essential COM where he is building a QueryInterface function using interface tables and the above macro is somehow used to find the pointer to the interface vtable of the class, where class is the ClassName implementing the BaseName, although I dont know how it is doing that.
It tells to the compiler: "imagine there a ClassName object at 0x10000000. Where would the BaseName data begin in that object, relative to 0x10000000"?
Think of a memory layout of a class object with multiple bases:
class A: B, C{};
In the memory block that constitutes an A object, there's the chunk of data that belong to B, also a chunk of data that belongs to C, and the data that are specific to A. Since the address of at least one base's data cannot be the same as the address of the class instance as a whole, the numeric value of the this pointer that you pass to different methods needs to vary. The macro retrieves the value of the difference.
EDIT: The pointer to the vtable is, by convention, the first data member in any class with virtual functions. So by finding the address of the base data, one finds the address of its vtable pointer.
Now, about the type conversion. Normally, when you typecast pointers, the operation is internally trivial - the numeric value of the address does not depend on what type does it point to; the very notion of datatype only exists on the C level. There's one important exception though - when you cast object pointers with multiple inheritance. As we've just discussed, the this pointer that you need to pass to a base class method might be numerically different from the one of the derived object's.
So the distinction between static_cast and reinterpret_cast captures this difference neatly. When you use reinterpret_cast, you're telling the compiler: "I know better. Take this numeric value and interpret it as a pointer to what I say". This is a deliberate subversion of the type system, dangerous, but occasionally necessary. This kind of cast is by definition trivial - cause you say so.
By "trivial" I mean - the numeric value of the pointer does not change.
The static_cast is a more high level construct. In this particular case, you're casting between an object and its base. That's a reasonable, safe cast under C++ class rules - BUT it might be numerically nontrivial. That's why the macro uses two different typecasts. static_cast does NOT violate the type system.
To recap:
reinterpret_cast<ClassName* >(OxlOOOOOOO)
is an unsafe operation. It returns a bogus pointer to a bogus object, but it's OK because we never dereference it.
static_cast<BaseName*>(...)
is a safe operation (with an unsafe pointer, the irony). It's the part where the nontrivial pointer typecast happens.
(DWORD(...)-OxlOOOOOOO)
is pure arithmetic. That's where the unsafety doubles back on itself: rather than use the pointer as a pointer, we cast it back to an integer and forget that it ever was a pointer.
The last stage could be equivalently rephrased as:
((char*)(...)-(char*)OxlOOOOOOO)
if that makes more sense.
Remark about magic 0x10000000 constant.
If that constant will be 0, GCC will show warning -Winvalid-offset-of (if it is enabled, of course). Maybe other compilers do something like that.
(I've edited this question to avoid distractions. There is one core question which would need to be cleared up before any other question would make sense. Apologies to anybody whose answer now seems less relevant.)
Let's set up a specific example:
struct Base {
int i;
};
There are no virtual method, and there is no inheritance, and is generally a very dumb and simple object. Hence it's Plain Old Data (POD) and it falls back on a predictable layout. In particular:
Base b;
&b == reinterpret_cast<B*>&(b.i);
This is according to Wikipedia (which itself claims to reference the C++03 standard):
A pointer to a POD-struct object, suitably converted using a reinterpret cast, points to its initial member and vice versa, implying that there is no padding at the beginning of a POD-struct.[8]
Now let's consider inheritance:
struct Derived : public Base {
};
Again, there are no virtual methods, no virtual inheritance, and no multiple inheritance. Therefore this is POD also.
Question: Does this fact (Derived is POD in C++11) allow us to say that:
Derived d;
&d == reinterpret_cast<D*>&(d.i); // true on g++-4.6
If this is true, then the following would be well-defined:
Base *b = reinterpret_cast<Base*>(malloc(sizeof(Derived)));
free(b); // It will be freeing the same address, so this is OK
I'm not asking about new and delete here - it's easier to consider malloc and free. I'm just curious about the regulations about the layout of derived objects in simple cases like this, and where the initial non-static member of the base class is in a predictable location.
Is a Derived object supposed to be equivalent to:
struct Derived { // no inheritance
Base b; // it just contains it instead
};
with no padding beforehand?
You don't care about POD-ness, you care about standard-layout. Here's the definition, from the standard section 9 [class]:
A standard-layout class is a class that:
has no non-static data members of type non-standard-layout class (or array of such types) or reference,
has no virtual functions (10.3) and no virtual base classes (10.1),
has the same access control (Clause 11) for all non-static data members,
has no non-standard-layout base classes,
either has no non-static data members in the most derived class and at most one base class with non-static data members, or has no base classes with non-static data members, and
has no base classes of the same type as the first non-static data member.
And the property you want is then guaranteed (section 9.2 [class.mem]):
A pointer to a standard-layout struct object, suitably converted using a reinterpret_cast, points to its initial member (or if that member is a bit-field, then to the unit in which it resides) and vice versa.
This is actually better than the old requirement, because the ability to reinterpret_cast isn't lost by adding non-trivial constructors and/or destructor.
Now let's move to your second question. The answer is not what you were hoping for.
Base *b = new Derived;
delete b;
is undefined behavior unless Base has a virtual destructor. See section 5.3.5 ([expr.delete])
In the first alternative (delete object), if the static type of the object to be deleted is different from its dynamic type, the static type shall be a base class of the dynamic type of the object to be deleted and the static type shall have a virtual destructor or the behavior is undefined.
Your earlier snippet using malloc and free is mostly correct. This will work:
Base *b = new (malloc(sizeof(Derived))) Derived;
free(b);
because the value of pointer b is the same as the address returned from placement new, which is in turn the same address returned from malloc.
Presumably your last bit of code is intended to say:
Base *b = new Derived;
delete b; // delete b, not d.
In that case, the short answer is that it remains undefined behavior. The fact that the class or struct in question is POD, standard layout or trivially copyable doesn't really change anything.
Yes, you're passing the right address, and yes, you and I know that in this case the dtor is pretty much a nop -- nonetheless, the pointer you're passing to delete has a different static type than dynamic type, and the static type does not have a virtual dtor. The standard is quite clear that this gives undefined behavior.
From a practical viewpoint, you can probably get away with the UB if you really insist -- chances are pretty good that there won't be any harmful side effects from what you're doing, at least with most typical compilers. Beware, however, that even at best the code is extremely fragile so seemingly trivial changes could break everything -- and even switching to a compiler with really heavy type checking and such could do so as well.
As far as your argument goes, the situation's pretty simple: it basically means the committee probably could make this defined behavior if they wanted to. As far as I know, however, it's never been proposed, and even if it had it would probably be a very low priority item -- it doesn't really add much, enable new styles of programming, etc.
This is meant as a supplement to Ben Voigt's answer', not a replacement.
You might think that this is all just a technicality. That the standard calling it 'undefined' is just a bit of semantic twaddle that has no real-world effects beyond allowing compiler writers to do silly things for no good reason. But this is not the case.
I could see desirable implementations in which:
Base *b = new Derived;
delete b;
Resulted in behavior that was quite bizarre. This is because storing the size of your allocated chunk of memory when it is known statically by the compiler is kind of silly. For example:
struct Base {
};
struct Derived {
int an_int;
};
In this case, when delete Base is called, the compiler has every reason (because of the rule you quoted at the beginning of your question) to believe that the size of the data pointed at is 1, not 4. If it, for example, implements a version of operator new that has a separate array in which 1 byte entities are all densely packed, and a different array in which 4 byte entities are all densely packed, it will end up assuming the Base * points to somewhere in the 1-byte entity array when in fact it points somewhere in the 4-byte entity array, and making all kinds of interesting errors for this reason.
I really wish operator delete had been defined to also take a size, and the compiler passed in either the statically known size if operator delete was called on an object with a non-virtual destructor, or the known size of the actual object being pointed at if it were being called as a result of a virtual destructor. Though this would likely have other ill effects and maybe isn't such a good idea (like if there are cases in which operator delete is called without a destructor having been called). But it would make the problem painfully obvious.
There is lots of discussion on irrelevant issues above. Yes, mainly for C compatibility there are a number of guarantees you can rely as long as you know what you are doing. All this is, however, irrelevant to your main question. The main question is: Is there any situation where an object can be deleted using a pointer type which doesn't match the dynamic type of the object and where the pointed to type doesn't have a virtual destructor. The answer is: no, there is not.
The logic for this can be derived from what the run-time system is supposed to do: it gets a pointer to an object and is asked to delete it. It would need to store information on how to call derived class destructors or about the amount of memory the object actually takes if this were to be defined. However, this would imply a possibly quite substantial cost in terms of used memory. For example, if the first member requires very strict alignment, e.g. to be aligned at an 8 byte boundary as is the case for double, adding a size would add an overhead of at least 8 bytes to allocate memory. Even though this might not sound too bad, it may mean that only one object instead of two or four fits into a cache line, reducing performance substantially.
Can you please explain what's going on in this buggy example:
Base base; Derived* d = reinterpret_cast<Derived*> (&base);
d->method();
d->virtual_method();
//output: Derived-method() Base-virtual_method()
I would expect this code to behave the other way around. Probably the compiler shares a single memory layout for Base and Derived, and of course the vtable is common.
When d->method is invoked I would expect the compiler to say "I'm just calling method at offset 0 with respect to my pointer. The pointer points to Base object, the only object around.
When d->virtual_method is invoked the compiler should say I am going to resolve it through the vtable, and thus the Derived method should be called (though the Base object is the only one around, the layout extends to Derived).
So I am expecting to see:
//output: Base-method() Derived-virtual_method()
base is a Base object; you reinterpret its bytes as a Derived object and then attempt to use it as if it were a Derived object. The behavior when you do this is undefined. Your program might crash; it might appear to do the right thing; it might make your computer light on fire.
Note that it is never correct to use reinterpret_cast to cast up and down a class hierarchy. You must use static_cast or dynamic_cast (or, if you are converting to a base class, no cast may be necessary).
To explain why you see this particular behavior, though: when you call a nonvirtual member function (as you do with d->method(), assuming method is a nonvirtual member function of Derived), the function that gets called is determined at compile time, not at runtime.
Here, the compiler knows that d points to a D object (because you've lied to the compiler and said that it is), so it generates code that calls Derived::method(). There is no "offset with respect to a pointer" at all. No computation needs to be done because the function to be called is known when the program is compiled.
Only when you call a virtual member function is a table lookup required (and even then, the lookup is only required when the compiler doesn't know the dynamic type of the object on which the member function is being called).
When you call d->virtual_method(), Base::virtual_method gets called. Why? In this particular implementation of C++, the first few bytes of an object of a class type that has virtual member functions (a polymorphic class type) contain a tag (called a "vptr" or a "virtual table pointer") that identifies the actual type of the object. When you call a virtual member function, then at runtime that tag is inspected and the function that is called is selected based on that tag.
When you reinterpret base as a Derived object, you don't actually change the object itself, so its tag still states that it is a Base object, hence why Base::virtual_method gets called.
Remember, though, that all this just happens to be what happens when you compile this code with a particular version of a particular compiler. The behavior is undefined and this is just one way that the undefined behavior can manifest itself.
The compiler only allocates enough memory to hold the requested object. Base might be 20 bytes, and Derived might be an extra 10 bytes on top of that (so Derived is 30 bytes in size.)
When you allocate 20 bytes for Base, and then (via Derived) access byte position 25, it's past the end of the allocated memory and (at best) you will get a crash.
The compiler cannot allocate 30 bytes for Base as you suggest, because not only would this be wasteful, but Derived could be implemented in a third party library and it may not even be known about when Base is being compiled.
How does the conversion between derived and base class internally occurs and how does compiler knows or does it store the size of object?
For example in the following:
class A
{
public:
A():x(2){};
private:
int x;
};
class B : public A
{
public:
B():A(),y(5){};
private:
int y;
};
class C : public B
{
public:
C():B(),z(9){};
private:
int z;
};
int main()
{
C *CObj = new C;
B *pB = static_cast<B*>(CObj);
delete CObj;
}
Edit: It must have been this:
B BObj = static_cast<B>(*CObj);
You don't have any "derived to base" conversion in your code. What you have in your code is a pointer-to-derived to pointer-to-base conversion. (This conversion does not require any explicit cast, BTW)
B *pB = CObj; // no need for the cast
In order to perform the pointer conversion, there's no need to know the size of the object. So, it is not clear where your reference to "size of the object" comes from.
In fact, in the typical implementation the above conversion for single-inheritance hierarchy of non-polymorphic classes is purely conceptual. I.e. the compiler does not do anything besides simply copying the numerical value of the derived pointer into the base pointer. No extra information is needed to perform this operation. No size, no nothing.
In more complicated situations (like multiple inheritance), the compiler might indeed have to generate code that would adjust the value of the pointer. And it will indeed need to know the sizes of the objects involved. But the sizes involved are always compile-time ones, i.e. they are compile-time constants, meaning that the compiler does immediately know them.
In even more complicated cases, like virtual inheritance, this conversion is normally supported by run-time structures implicitly built into the object, which will include everything deemed necessary. Run-time size of the object might be included as well, if the implementation chooses to do so.
Note that you don't need the static_cast here; it's perfectly legal to "up-cast" a pointer-to-derived-class to a pointer-to-parent-class.
In this example, there is no conversion going on. The pointer value stays the same (i.e. under the hood, CObj and pB point at the same memory, though things get more complex with multiple inheritance). The compiler organises the members of B and C objects in memory so that everything just works. As we're dealing with pointers, the size of the object doesn't matter (that was only relevant when you created a new C).
If you had any virtual methods, then we could talk about vtables and vptrs (http://en.wikipedia.org/wiki/Vtable).
A derived class object has base class subobjects. Specifically the Standard says in 10.3
"The order in which the base class
subobjects are allocated in the most
derived object (1.8) is unspecified"
This means that even though many a times, the base subobject could be right at the beginning of the derived object, it is not necessary. Hence the conversion from Derived* to Base* is completely unspecified and is probably left as a degree of latitude to compiler developers.
I would say that it is important to know the rules of the language and the reason behind the same, rather than worry about how compiler implements them. As an example, I have seen far too many discussions on VTABLE and VPTR which is a compiler specific implementation to achieve dynamic binding. Instead it helps to know about the concept of 'unique final overrider' that is enough to understand the concept of virtual functions and dynamic binding. The point is to focus on 'what' rather than 'how', because 'how' most of the times is not required. I say most of the times because in some cases it helps. An example is to understand the concept of 'pointer to members'. It helps to know that it is usually implemented in some form of 'offset' rather than being a regular pointer.
How does the conversion between derived and base class internally occurs
Implementation defined.
Imposable to answer unless you tell us which compiler you are using.
But generally not worth knowing or worrying about (unless you are writing a compiler).
and how does compiler knows [editor] size of the object
The compiler knows the size (It has worked out the size of C during compilation).
or does it store the size of object?
The object does not need to know the size and thus it is not stored as part of the class.
The runtime memory management (used via new) may need to know (but it is implementation defined) so that it can correctly release the memory (but anything it stores will not be stroed in the object).
If you have ever done any C, the answer would come from itself.
A memory allocator doesn't care at all about what it is storing. It just have to know what memory ranges has been allocated. It doesn't see the difference between a C and an int[4]. It just have to know how to free the memory range that starts at the given pointer.