memcpy derived class to base class, why still called base class function - c++

I am reading Inside the C++ Object Model. In section 1.3
So, then, why is it that, given
Bear b;
ZooAnimal za = b;
// ZooAnimal::rotate() invoked
za.rotate();
the instance of rotate() invoked is the ZooAnimal instance and not that of Bear? Moreover, if memberwise initialization copies the values of one object to another, why is za's vptr not addressing Bear's virtual table?
The answer to the second question is that the compiler intercedes in the initialization and assignment of one class object with another. The compiler must ensure that if an object contains one or more vptrs, those vptr values are not initialized or changed by the source object .
So I wrote the test code below:
#include <stdio.h>
class Base{
public:
virtual void vfunc() { puts("Base::vfunc()"); }
};
class Derived: public Base
{
public:
virtual void vfunc() { puts("Derived::vfunc()"); }
};
#include <string.h>
int main()
{
Derived d;
Base b_assign = d;
Base b_memcpy;
memcpy(&b_memcpy, &d, sizeof(Base));
b_assign.vfunc();
b_memcpy.vfunc();
printf("sizeof Base : %d\n", sizeof(Base));
Base &b_ref = d;
b_ref.vfunc();
printf("b_assign: %x; b_memcpy: %x; b_ref: %x\n",
*(int *)&b_assign,
*(int *)&b_memcpy,
*(int *)&b_ref);
return 0;
}
The result
Base::vfunc()
Base::vfunc()
sizeof Base : 4
Derived::vfunc()
b_assign: 80487b4; b_memcpy: 8048780; b_ref: 8048780
My question is why b_memcpy still called Base::vfunc()

What you are doing is illegal in C++ language, meaning that the behavior of your b_memcpy object is undefined. The latter means that any behavior is "correct" and your expectations are completely unfounded. There's not much point in trying to analyze undefined behavior - it is not supposed to follow any logic.
In practice, it is quite possible that your manipulations with memcpy did actually copy Derived's virtual table pointer to b_memcpy object. And your experiments with b_ref confirm that. However, when a virtual method is called though an immediate object (as is the case with b_memcpy.vfunc() call) most implementations optimize away the access to the virtual table and perform a direct (non-virtual) call to the target function. Formal rules of the language state that no legal action can ever make b_memcpy.vfunc() call to dispatch to anything other than Base::vfunc(), which is why the compiler can safely replace this call with a direct call to Base::vfunc(). This is why any virtual table manipulations will normally have no effect on b_memcpy.vfunc() call.

The behavior you've invoked is undefined because the standard says it's undefined, and your compiler takes advantage of that fact. Lets look at g++ for a concrete example. The assembly it generates for the line b_memcpy.vfunc(); with optimizations disabled looks like this:
lea rax, [rbp-48]
mov rdi, rax
call Base::vfunc()
As you can see, the vtable wasn't even referenced. Since the compiler knows the static type of b_memcpy it has no reason to dispatch that method call polymorphically. b_memcpy can't be anything other than a Base object, so it just generates a call to Base::vfunc() as it would with any other method call.
Going a bit further, lets add a function like this:
void callVfunc(Base& b)
{
b.vfunc();
}
Now if we call callVfunc(b_memcpy); we can see different results. Here we get a different result depending on the optimization level at which I compile the code. On -O0 and -O1 Derived::vfunc() is called and on -O2 and -O3 Base::vfunc() is printed. Again, since the standard says the behavior of your program is undefined, the compiler makes no effort to produce a predictable result, and simply relies on the assumptions made by the language. Since the compiler knows b_memcpy is a Base object, it can simply inline the call to puts("Base::vfunc()"); when the optimization level allows for it.

You aren't allowed to do
memcpy(&b_memcpy, &d, sizeof(Base));
- it's undefined behaviour, because b_memcpy and d aren't "plain old data" objects (because they have virtual member functions).
If you wrote:
b_memcpy = d;
then it would print Base::vfunc() as expected.

Any use of a vptr is outside the scope of the standard
Granted, the use of memcpy here has UB
The answers pointing out that any use of memcpy, or other byte manipulation of non-PODs, that is, of any object with a vptr, has undefined behavior, are strictly technically correct but do not answer the question. The question is predicated on the existence of a vptr (vtable pointer) which isn't even mandated by the standard: of course the answer will involve facts outside the standard and the result bill not be guaranteed by the standard!
Standard text is not relevant regarding the vptr
The issue is not that you are not allowed to manipulate the vptr; the notion of being allowed by the standard to manipulate anything that is not even described in the standard text is absurd. Of course not standard way to change the vptr will exist and this is beside the point.
The vptr encodes the type of a polymorphic object
The issue here is not what the standard says about the vptr, the issue is what the vptr represents, and what the standard says about that: the vptr represents the dynamic type of an object. Whenever the result of an operation depends on the dynamic type, the compiler will generate code to use the vptr.
[Note regarding MI: I say "the" vptr (as if the only one vptr), but when MI (multiple inheritance) is involved, objects can have more than one vptr, each representing the complete object viewed as a particular polymorphic base class type. (A polymorphic class is a class with a least one virtual function.)]
[Note regarding virtual bases: I mention only the vptr, but some compilers insert other pointers to represent aspects of the dynamic type, like the location of virtual base subobjects, and some other compilers use the vptr for that purpose. What is true about the vptr is also true about these other internal pointers.]
So a particular value of the vptr corresponds to a dynamic type: that is the type of most derived object.
Changes of the dynamic type of an object during its lifetime
During construction, the dynamic type changes, and that is why virtual function calls from inside the constructor can be "surprising". Some people say that the rules of calling virtual functions during construction are special, but they are absolutely not: the final overrider is called; that override is the one the class corresponding to the most derived object that has been constructed, and in a constructor C::C(arg-list), it is always the type of the class C.
During destruction, the dynamic type changes, in the reverse order. Calls to virtual function from inside destructors follow the same rules.
What it means when something is left undefined
You can do low level manipulations that are not sanctioned in the standard. That a behavior is not explicitly defined in the C++ standard does not imply that it is not described elsewhere. Just because the result of a manipulation is explicitly described has having UB (undefined behavior) in the C++ standard does not mean your implementation cannot define it.
You can also use your knowledge of the way the compilers work: if strict separate compilation is used, that is when the compiler can get no information from separately compiled code, every separately compiled function is a "black box". You can use this fact: the compiler will have to assume that anything that a separately compiled function could do will be done. Even with inside a given function, you can use asm directive to get the same effects: an asm directive with no constraint can do anything that is legal in C++. The effect is a "forget what you know from code analysis at that point" directive.
The standard describes what can change the dynamic type, and nothing is allowed to change it except construction/destruction, so only an "external" (blackbox) function is is otherwise allowed to perform construction/destruction can change a dynamic type.
Calling constructors on an existing object is not allowed, except to reconstruct it with the exact same type (and with restrictions) see [basic.life]/8 :
If, after the lifetime of an object has ended and before the storage
which the object occupied is reused or released, a new object is
created at the storage location which the original object occupied, a
pointer that pointed to the original object, a reference that referred
to the original object, or the name of the original object will
automatically refer to the new object and, once the lifetime of the
new object has started, can be used to manipulate the new object, if:
(8.1) the storage for the new object exactly overlays the storage
location which the original object occupied, and
(8.2) the new object is of the same type as the original object
(ignoring the top-level cv-qualifiers), and
(8.3) the type of the original object is not const-qualified, and, if
a class type, does not contain any non-static data member whose type
is const-qualified or a reference type, and
(8.4) the original object was a most derived object ([intro.object])
of type T and the new object is a most derived object of type T (that
is, they are not base class subobjects).
This means that the only case where you could call a constructor (with placement new) and still use the same expressions that used to designate the objects (its name, pointers to it, etc.) are those where the dynamic type would not change, so the vptr would still be the same.
On other words, if you want to overwrite the vptr using low level tricks, you could; but only if you write the same value.
On other words, don't try to hack the vptr.

Related

Is a data race on vptr explicitly illegal?

Before you go further, a note: this is purely a language lawyer question. I wish to get answers based on standard quotes. I am not looking for advice on writing C++ code. Please answer as if I was a compiler writer.
During construction of an object with only exclusive subobjects (#), notably those only non virtual bases (also those with a virtual base class named only once), the dynamic type of an lvalue referring to a base class subobject "increases": it goes from type of the base to type of the class of constructor running.
(#) A subobject is exclusive when it is a direct subobject of exactly one other object (which may be another subobject or a complete object). A member and a non virtual base are always exclusive.
During destruction, the type decreases (until the end of the body of the destructor of that subobject, where the subobject is gone and has no dynamic type anymore).
[During construction of an object with shared base class subobjects (that is in a class with distinct base subobjects with at least a virtual base), the dynamic type of a base subobject can "disappear" temporarily. I'm do not wish to discuss such classes here.]
The real question is: What happens if the dynamic type of the object is increased in another thread?
The title of the question, which is standard C++ question, is expressed using a non standard term (vptr), which may look contradicting. The reasons are:
There is no requirement that polymorphism is implemented in term of vptr, but it (almost?) always is. The one (or many) vptr in an object represent the dynamic type of a polymorphic object.
Data races are defined in term of read/write operations to a memory location.
The standard text often uses non standard elements "for exposition only" to define standard features. (So, why not use the vptr "for exposition only"?)
The standard does not define the behavior of polymorphic objects (*) directly as a function of their dynamic type; the standard specifies which expressions are allowed during the so-called "lifetime" (after the constructor has completed), inside the body of the constructor of the most derived type (exactly the same expressions are allowed with the same semantic), also inside the base class subobject constructors...
(*) Dynamic behavior of polymorphic or dynamic objects(**) include: virtual calls, derived to base conversions, down casts (static_cast or dynamic_cast), typeid of a polymorphic object.
(**) A dynamic object is one such that its class uses the virtual keyword; its constructor is not trivial for that reason.
So the description says: After something has finished, as soon as something started, before something else, etc. some expression is valid and does such and such.
The specification of construction and destruction was written before threads were part of standard C++. So what was the change with the standardization of threads? There is one sentence with defines threading behavior (the normative part) [basic.life]/11:
In this subclause, “before” and “after” refer to the “happens before”
relation ([intro.multithread]).
So it's clear that an object is seen as fully constructed iff there is an happen before relation between the completion of the invocation of the constructor and the use of the object, and also an happen before that use of the object and the invocation of the destructor (if it's invoked at all).
But it doesn't say what happens during the construction of derived classes, after a base class subobject has been constructed: obviously there is a race condition if any dynamic property is used for a polymorphic object under construction, but race conditions are not illegal.
[A race condition is a case of non-determinism, and any meaningful use of a mutex, condition variable, rwlocks, many uses of semaphores, many uses of other synchronisation devices, and all uses of atomic primitives introduce a race condition at least at the level of the modification order on the atomic object. Whether that low level non-determinism results on unpredictable high level behavior depends on the way the primitives are used.]
Then the standard draft goes on to say:
[ Note: Therefore, undefined behavior results if an object that is
being constructed in one thread is referenced from another thread
without adequate synchronization. — end note ]
Where is "adequate synchronization" defined?
Is the lack of "adequate synchronization" the moral equivalent of a regular data race: a data race on the vptr, or in standard speak, a data race on the dynamic type?
For simplicity, I wish to restrict the scope of the question to single inheritance, at least as a first step. (The standard is awfully confused about the construction of objects with multiple inheritance anyway.)
This is language lawyer question so I'm not interested in:
whether using an object that is in the process of being constructed in another thread is advisable (it's probably not advisable);
how to use synchronization to reliably fix that race condition;
whether compiler vendors wish to support such a use case (they probably do not and will not);
whether that could possibly work reliably in any real world implementation (it probably will not reliably work in non trivial cases with current implementation).
EDIT: The previous example, instead of illustrating the issue, was a distraction. It caused a very interesting but completely irrelevant discussion in the chat section.
Here is a cleaner example that will not cause the same issue:
atomic<Base1*> shared;
struct Base1 {
virtual void f() {}
};
struct Base2 : Base1 {
virtual void f() {}
Base2 () { shared = (Base1*)this; }
};
struct Der2 : Base2 {
virtual void f() {}
};
void use_shared() {
Base1 *p;
while (! (p = shared.get()));
p->f();
}
With the consumer/producer logic:
Thread A: new Der2;
Thread B: use_shared();
For reference, original example:
atomic<Base*> shared;
struct Base {
virtual void f() {}
Base () { shared = this; }
};
struct Der : Base {
virtual void f() {}
};
void use_shared() {
Base *p;
while (! (p = shared.get()));
p->f();
}
Consumer/producer logic:
Thread A: new Der;
Thread B: use_shared();
It wasn't clear that this could be used by another thread during the execution of Base constructor, which is an interesting issue but irrelevant to the issue of using a base class subobject while a derived constructor runs in another thread.
Additional information
For reference, the DR that "motivated" the current phrasing (although that explains nothing):
Core Language Defect Report #710
My reading of the standard is that there's a data race and therefore undefined behavior, but the standard addresses it very indirectly.
[basic.life]/1 The lifetime of an object of type T begins when ... its initialization is complete.
When shared = this; is executed, the lifetime of Base object, let alone Der, hasn't started yet.
[basic.life]/6 Before the lifetime of an object has started but after the storage which the object will occupy has been allocated ... any pointer that represents the address of the storage location where the object will be or was located may be used but only in limited ways. For an object under construction or destruction, see [class.cdtor]. Otherwise ... [t]he program has undefined behavior if ... the pointer is used to access a non-static data member or call a non-static member function of the object.
[basic.life]/11 In this section, “before” and “after” refer to the “happens before” relation (4.7). [ Note: Therefore, undefined behavior results if an object that is being constructed in one thread is referenced from another thread without
adequate synchronization. —end note ]
So the default position of [basic.life] is that a call to an object's method that doesn't happen-after its initialization is completed exhibits undefined behavior. But [class.cdtor] may have more to say.
[class.cdtor]/3 Member functions, including virtual functions (13.3), can be called during construction or destruction (15.6.2). When a virtual function is called directly or indirectly from a constructor or from a destructor ...
Thus, [class.cdtor] only addresses the case where the virtual function is called directly or indirectly from the constructor (necessarily on the same thread on which the constructor itself runs). It's silent on the case where a method is called from another thread, as in the example. I take it to mean that [basic.life] controls, and the behavior of the example is undefined.

Memory overhead of wrapper classes

When using wrapper classes in C++, like
class myInt {
int _value;
public:
myInt( int value ) : _value( value );
int value() const { return _value; }
}
, is there any memory overhead compared to a simple int?
This answer says "not if there's no virtual functions", but I want to understand the exact reason.
I looked into § 10.3 [class.virtual] of ISO/IEC 14882:2003, which say: "A class that declares or inherits a virtual function is called a polymorphic class." I understand that objects of such a class do not need to have a vtable pointer in them.
But I don't understand how it follows from this that it does not have to have any memory overhead. I can create a class deriving from myInt, calling it, say, myDerivedInt. Can't I do a dynamic_cast from a myInt * to a myDerivedInt *? If not, why not?
Is there any memory overhead compared to a simple €int`?
I would say not, but I don't know if it is guaranteed.
I understand that objects of such a class do not need to have a vtable pointer in them.
If the polymorphic class object is created on the stack, and its address is never taken, the compiler might figure out that its functions are never called virtually (dynamic-dispatch) and could remove the virtual table pointer from the stack. (This applies only on a per-case basis. Instances of the class in other places may include the vtable pointer). This kind of optimization would have very small impact, so I wouldn't worry about if it does or not. In general, a class which inherits from a class with a vtable, will also have a vtable pointer in its instances.
Can't I do a dynamic_cast from a myInt* to a myDerivedInt*?
No. Not if it doesn't have a vtable. In other words, if myInt doesn't have any virtual functions (or doesn't inherit from any class that does), then you cannot dynamic_cast from MyInt* to MyDerivedInt*. You can use static_cast, however.
If not, why not?
Because there is no vtable in myInt. The vtable stores the information needed for dynamic_cast.
But I don't understand how it follows from this that it does not have to have any memory overhead
Well, the only reason why memory overhead would be required is if something else has to be stored in the class. It has to contain an int, obviously, because that's its member. But it doesn't need a vtable pointer, and nothing else in the C++ standard mandates that anything else must be contained in the class. The class can be implemented without memory overhead not because the C++ standard says "this can be done with no memory overhead", but because it doesn't say "in addition to the int member, the class must also contain ". It doesn't specify anything that would require overhead to be introduced, so because of that, the overhead can be avoided.
And of course, when compilers can avoid additional overhead, they typically do so, so in practice such a class will have no memory overhead.
OK, while writing this and finishing the last sentence, I found the answer:
Clause 6 of § 5.2.7 [expr.dynamic.cast] says:
Otherwise, v shall be a pointer to or an lvalue of a polymorphic type (10.3).
I had actually looked "polymorphic type" in the index, but that only mentioned the occurence in 10.3 ...

PODs and inheritance in C++11. Does the address of the struct == address of the first member?

(I've edited this question to avoid distractions. There is one core question which would need to be cleared up before any other question would make sense. Apologies to anybody whose answer now seems less relevant.)
Let's set up a specific example:
struct Base {
int i;
};
There are no virtual method, and there is no inheritance, and is generally a very dumb and simple object. Hence it's Plain Old Data (POD) and it falls back on a predictable layout. In particular:
Base b;
&b == reinterpret_cast<B*>&(b.i);
This is according to Wikipedia (which itself claims to reference the C++03 standard):
A pointer to a POD-struct object, suitably converted using a reinterpret cast, points to its initial member and vice versa, implying that there is no padding at the beginning of a POD-struct.[8]
Now let's consider inheritance:
struct Derived : public Base {
};
Again, there are no virtual methods, no virtual inheritance, and no multiple inheritance. Therefore this is POD also.
Question: Does this fact (Derived is POD in C++11) allow us to say that:
Derived d;
&d == reinterpret_cast<D*>&(d.i); // true on g++-4.6
If this is true, then the following would be well-defined:
Base *b = reinterpret_cast<Base*>(malloc(sizeof(Derived)));
free(b); // It will be freeing the same address, so this is OK
I'm not asking about new and delete here - it's easier to consider malloc and free. I'm just curious about the regulations about the layout of derived objects in simple cases like this, and where the initial non-static member of the base class is in a predictable location.
Is a Derived object supposed to be equivalent to:
struct Derived { // no inheritance
Base b; // it just contains it instead
};
with no padding beforehand?
You don't care about POD-ness, you care about standard-layout. Here's the definition, from the standard section 9 [class]:
A standard-layout class is a class that:
has no non-static data members of type non-standard-layout class (or array of such types) or reference,
has no virtual functions (10.3) and no virtual base classes (10.1),
has the same access control (Clause 11) for all non-static data members,
has no non-standard-layout base classes,
either has no non-static data members in the most derived class and at most one base class with non-static data members, or has no base classes with non-static data members, and
has no base classes of the same type as the first non-static data member.
And the property you want is then guaranteed (section 9.2 [class.mem]):
A pointer to a standard-layout struct object, suitably converted using a reinterpret_cast, points to its initial member (or if that member is a bit-field, then to the unit in which it resides) and vice versa.
This is actually better than the old requirement, because the ability to reinterpret_cast isn't lost by adding non-trivial constructors and/or destructor.
Now let's move to your second question. The answer is not what you were hoping for.
Base *b = new Derived;
delete b;
is undefined behavior unless Base has a virtual destructor. See section 5.3.5 ([expr.delete])
In the first alternative (delete object), if the static type of the object to be deleted is different from its dynamic type, the static type shall be a base class of the dynamic type of the object to be deleted and the static type shall have a virtual destructor or the behavior is undefined.
Your earlier snippet using malloc and free is mostly correct. This will work:
Base *b = new (malloc(sizeof(Derived))) Derived;
free(b);
because the value of pointer b is the same as the address returned from placement new, which is in turn the same address returned from malloc.
Presumably your last bit of code is intended to say:
Base *b = new Derived;
delete b; // delete b, not d.
In that case, the short answer is that it remains undefined behavior. The fact that the class or struct in question is POD, standard layout or trivially copyable doesn't really change anything.
Yes, you're passing the right address, and yes, you and I know that in this case the dtor is pretty much a nop -- nonetheless, the pointer you're passing to delete has a different static type than dynamic type, and the static type does not have a virtual dtor. The standard is quite clear that this gives undefined behavior.
From a practical viewpoint, you can probably get away with the UB if you really insist -- chances are pretty good that there won't be any harmful side effects from what you're doing, at least with most typical compilers. Beware, however, that even at best the code is extremely fragile so seemingly trivial changes could break everything -- and even switching to a compiler with really heavy type checking and such could do so as well.
As far as your argument goes, the situation's pretty simple: it basically means the committee probably could make this defined behavior if they wanted to. As far as I know, however, it's never been proposed, and even if it had it would probably be a very low priority item -- it doesn't really add much, enable new styles of programming, etc.
This is meant as a supplement to Ben Voigt's answer', not a replacement.
You might think that this is all just a technicality. That the standard calling it 'undefined' is just a bit of semantic twaddle that has no real-world effects beyond allowing compiler writers to do silly things for no good reason. But this is not the case.
I could see desirable implementations in which:
Base *b = new Derived;
delete b;
Resulted in behavior that was quite bizarre. This is because storing the size of your allocated chunk of memory when it is known statically by the compiler is kind of silly. For example:
struct Base {
};
struct Derived {
int an_int;
};
In this case, when delete Base is called, the compiler has every reason (because of the rule you quoted at the beginning of your question) to believe that the size of the data pointed at is 1, not 4. If it, for example, implements a version of operator new that has a separate array in which 1 byte entities are all densely packed, and a different array in which 4 byte entities are all densely packed, it will end up assuming the Base * points to somewhere in the 1-byte entity array when in fact it points somewhere in the 4-byte entity array, and making all kinds of interesting errors for this reason.
I really wish operator delete had been defined to also take a size, and the compiler passed in either the statically known size if operator delete was called on an object with a non-virtual destructor, or the known size of the actual object being pointed at if it were being called as a result of a virtual destructor. Though this would likely have other ill effects and maybe isn't such a good idea (like if there are cases in which operator delete is called without a destructor having been called). But it would make the problem painfully obvious.
There is lots of discussion on irrelevant issues above. Yes, mainly for C compatibility there are a number of guarantees you can rely as long as you know what you are doing. All this is, however, irrelevant to your main question. The main question is: Is there any situation where an object can be deleted using a pointer type which doesn't match the dynamic type of the object and where the pointed to type doesn't have a virtual destructor. The answer is: no, there is not.
The logic for this can be derived from what the run-time system is supposed to do: it gets a pointer to an object and is asked to delete it. It would need to store information on how to call derived class destructors or about the amount of memory the object actually takes if this were to be defined. However, this would imply a possibly quite substantial cost in terms of used memory. For example, if the first member requires very strict alignment, e.g. to be aligned at an 8 byte boundary as is the case for double, adding a size would add an overhead of at least 8 bytes to allocate memory. Even though this might not sound too bad, it may mean that only one object instead of two or four fits into a cache line, reducing performance substantially.

Explain bug behaviour

Can you please explain what's going on in this buggy example:
Base base; Derived* d = reinterpret_cast<Derived*> (&base);
d->method();
d->virtual_method();
//output: Derived-method() Base-virtual_method()
I would expect this code to behave the other way around. Probably the compiler shares a single memory layout for Base and Derived, and of course the vtable is common.
When d->method is invoked I would expect the compiler to say "I'm just calling method at offset 0 with respect to my pointer. The pointer points to Base object, the only object around.
When d->virtual_method is invoked the compiler should say I am going to resolve it through the vtable, and thus the Derived method should be called (though the Base object is the only one around, the layout extends to Derived).
So I am expecting to see:
//output: Base-method() Derived-virtual_method()
base is a Base object; you reinterpret its bytes as a Derived object and then attempt to use it as if it were a Derived object. The behavior when you do this is undefined. Your program might crash; it might appear to do the right thing; it might make your computer light on fire.
Note that it is never correct to use reinterpret_cast to cast up and down a class hierarchy. You must use static_cast or dynamic_cast (or, if you are converting to a base class, no cast may be necessary).
To explain why you see this particular behavior, though: when you call a nonvirtual member function (as you do with d->method(), assuming method is a nonvirtual member function of Derived), the function that gets called is determined at compile time, not at runtime.
Here, the compiler knows that d points to a D object (because you've lied to the compiler and said that it is), so it generates code that calls Derived::method(). There is no "offset with respect to a pointer" at all. No computation needs to be done because the function to be called is known when the program is compiled.
Only when you call a virtual member function is a table lookup required (and even then, the lookup is only required when the compiler doesn't know the dynamic type of the object on which the member function is being called).
When you call d->virtual_method(), Base::virtual_method gets called. Why? In this particular implementation of C++, the first few bytes of an object of a class type that has virtual member functions (a polymorphic class type) contain a tag (called a "vptr" or a "virtual table pointer") that identifies the actual type of the object. When you call a virtual member function, then at runtime that tag is inspected and the function that is called is selected based on that tag.
When you reinterpret base as a Derived object, you don't actually change the object itself, so its tag still states that it is a Base object, hence why Base::virtual_method gets called.
Remember, though, that all this just happens to be what happens when you compile this code with a particular version of a particular compiler. The behavior is undefined and this is just one way that the undefined behavior can manifest itself.
The compiler only allocates enough memory to hold the requested object. Base might be 20 bytes, and Derived might be an extra 10 bytes on top of that (so Derived is 30 bytes in size.)
When you allocate 20 bytes for Base, and then (via Derived) access byte position 25, it's past the end of the allocated memory and (at best) you will get a crash.
The compiler cannot allocate 30 bytes for Base as you suggest, because not only would this be wasteful, but Derived could be implemented in a third party library and it may not even be known about when Base is being compiled.

Derived to base class conversion

How does the conversion between derived and base class internally occurs and how does compiler knows or does it store the size of object?
For example in the following:
class A
{
public:
A():x(2){};
private:
int x;
};
class B : public A
{
public:
B():A(),y(5){};
private:
int y;
};
class C : public B
{
public:
C():B(),z(9){};
private:
int z;
};
int main()
{
C *CObj = new C;
B *pB = static_cast<B*>(CObj);
delete CObj;
}
Edit: It must have been this:
B BObj = static_cast<B>(*CObj);
You don't have any "derived to base" conversion in your code. What you have in your code is a pointer-to-derived to pointer-to-base conversion. (This conversion does not require any explicit cast, BTW)
B *pB = CObj; // no need for the cast
In order to perform the pointer conversion, there's no need to know the size of the object. So, it is not clear where your reference to "size of the object" comes from.
In fact, in the typical implementation the above conversion for single-inheritance hierarchy of non-polymorphic classes is purely conceptual. I.e. the compiler does not do anything besides simply copying the numerical value of the derived pointer into the base pointer. No extra information is needed to perform this operation. No size, no nothing.
In more complicated situations (like multiple inheritance), the compiler might indeed have to generate code that would adjust the value of the pointer. And it will indeed need to know the sizes of the objects involved. But the sizes involved are always compile-time ones, i.e. they are compile-time constants, meaning that the compiler does immediately know them.
In even more complicated cases, like virtual inheritance, this conversion is normally supported by run-time structures implicitly built into the object, which will include everything deemed necessary. Run-time size of the object might be included as well, if the implementation chooses to do so.
Note that you don't need the static_cast here; it's perfectly legal to "up-cast" a pointer-to-derived-class to a pointer-to-parent-class.
In this example, there is no conversion going on. The pointer value stays the same (i.e. under the hood, CObj and pB point at the same memory, though things get more complex with multiple inheritance). The compiler organises the members of B and C objects in memory so that everything just works. As we're dealing with pointers, the size of the object doesn't matter (that was only relevant when you created a new C).
If you had any virtual methods, then we could talk about vtables and vptrs (http://en.wikipedia.org/wiki/Vtable).
A derived class object has base class subobjects. Specifically the Standard says in 10.3
"The order in which the base class
subobjects are allocated in the most
derived object (1.8) is unspecified"
This means that even though many a times, the base subobject could be right at the beginning of the derived object, it is not necessary. Hence the conversion from Derived* to Base* is completely unspecified and is probably left as a degree of latitude to compiler developers.
I would say that it is important to know the rules of the language and the reason behind the same, rather than worry about how compiler implements them. As an example, I have seen far too many discussions on VTABLE and VPTR which is a compiler specific implementation to achieve dynamic binding. Instead it helps to know about the concept of 'unique final overrider' that is enough to understand the concept of virtual functions and dynamic binding. The point is to focus on 'what' rather than 'how', because 'how' most of the times is not required. I say most of the times because in some cases it helps. An example is to understand the concept of 'pointer to members'. It helps to know that it is usually implemented in some form of 'offset' rather than being a regular pointer.
How does the conversion between derived and base class internally occurs
Implementation defined.
Imposable to answer unless you tell us which compiler you are using.
But generally not worth knowing or worrying about (unless you are writing a compiler).
and how does compiler knows [editor] size of the object
The compiler knows the size (It has worked out the size of C during compilation).
or does it store the size of object?
The object does not need to know the size and thus it is not stored as part of the class.
The runtime memory management (used via new) may need to know (but it is implementation defined) so that it can correctly release the memory (but anything it stores will not be stroed in the object).
If you have ever done any C, the answer would come from itself.
A memory allocator doesn't care at all about what it is storing. It just have to know what memory ranges has been allocated. It doesn't see the difference between a C and an int[4]. It just have to know how to free the memory range that starts at the given pointer.