vtables and this pointer - c++

I was trying to learn some more about the inner workings of vtables and vpointers, so I decided to try to access the vtable directly using some tricks. I created two classes, Base and Derv, each having two virtual functions (Derv overriding those of Base).
class Base
{
int x;
int y;
public:
Base(int x_, int y_) : x(x_), y(y_) {}
virtual void foo() { cout << "Base::foo(): x = " << x << '\n'; }
virtual void bar() { cout << "Base::bar(): y = " << y << '\n'; }
};
class Derv: public Base
{
int x;
int y;
public:
Derv(int x_, int y_) : Base(x_, y_), x(x_), y(y_) {}
virtual void foo() { cout << "Derived::foo(): x = " << x << '\n'; }
virtual void bar() { cout << "Derived::bar(): y = " << y << '\n'; }
};
Now, the compiler adds a vtable pointer to each class, occupying the first 4 bytes (32 bits) in memory. I accessed this pointer by casting the address of an object to a size_t*, since the pointer points to another pointer of size sizeof(size_t). The virtual functions can now be accessed by indexing the vpointer, and casting the result to a function pointer of the appropriate type. I encapsulated these steps in a function:
template <typename T>
void call(T *ptr, size_t num)
{
typedef void (*FunPtr)();
size_t *vptr = *reinterpret_cast<size_t**>(ptr);
FunPtr fun = reinterpret_cast<FunPtr>(vptr[num]);
//setThisPtr(ptr); added later, see below!
fun();
}
When one of the memberfunctions are called this way, e.g. call(new Base(1, 2), 0) to call Base::foo(), it is hard to predict what will happen, since they are called without a this-pointer. I solved this by adding a little templatized function, knowing that g++ stores the this-pointer in the ecx register (this however forces me to compile with the -m32 compiler flag):
template <typename T>
void setThisPtr(T *ptr)
{
asm ( mov %0, %%ecx;" :: "r" (ptr) );
}
Uncommenting the setThisPtr(ptr) line in the snippet above now makes it a working program:
int main()
{
Base* base = new Base(1, 2);
Base* derv = new Derv(3, 4);
call(base, 0); // "Base::foo(): x = 1"
call(base, 1); // "Base::bar(): y = 2"
call(derv, 0); // "Derv::foo(): x = 3"
call(derv, 1); // "Derv::bar(): y = 4"
}
I decided to share this, since in the process of writing this little program I gained more insight in how vtables work and it might help others in understanding this material a little better.
However I still have some questions:
1. Which register is used (gcc 4.x) to store the this-pointer when compiling a 64-bit binary? I tried all 64-bit registers as documented here: http://developers.sun.com/solaris/articles/asmregs.html
2. When/how is the this-pointer set? I suspect that the compiler sets the this pointer on each function call through an object in a similar way as to how I just did it. Is this the way polymorphism actually works? (By setting the this-pointer first, then calling the virtual function from the vtable?).

On Linux x86_64, and I believe other UNIX-like OSes, function calls follow the System V ABI (AMD64), which itself follows the IA-64 C++ ABI for C++. Depending on the method's type, the this pointer is either passed implicitly through first argument or the second argument (when the return value has non-trivial copy constructor or destructor, it must live as a temporary on stack, and the first argument is implicitly a pointer to that space); otherwise, virtual method calls are identical to function calls in C (integer/pointer arguments in %rdi, %rsi, %rdx, %rcx, %r8, %r9, overflowing to stack; integer/pointer return in %rax; floats in %xmm0-%xmm7; etc.). Virtual method dispatch works by looking up a pointer in the vtable then calling it just like a non-virtual method.
I'm less familiar with Windows x64 conventions, but I believe it to be similar in that C++ method calls follow the exact same structure as C function calls (which use different registers than on Linux), just with an implicit this argument first.

Related

When is the value of "this" shifted by an offset?

I was wondering whether assert( this != nullptr ); was a good idea in member functions and someone pointed out that it wouldn’t work if the value of this had been added an offset. In that case, instead of being 0, it would be something like 40, making the assert useless.
When does this happen though?
Multiple inheritance can cause an offset, skipping the extra v-table pointers in the object. The generic name is "this pointer adjustor thunking".
But you are helping too much. Null references are very common bugs, the operating system already has an assert built-in for you. Your program will stop with a segfault or access violation. The diagnostic you'll get from the debugger is always good enough to tell you that the object pointer is null, you'll see a very low address. Not just null, it works for MI cases as well.
this adjustment can happen only in classes that use multiple-inheritance. Here's a program that illustrates this:
#include <iostream>
using namespace std;
struct A {
int n;
void af() { cout << "this=" << this << endl; }
};
struct B {
int m;
void bf() { cout << "this=" << this << endl; }
};
struct C : A,B {
};
int main(int argc, char** argv) {
C* c = NULL;
c->af();
c->bf();
return 0;
}
When I run this program I get this output:
this=0
this=0x4
That is: your assert this != nullptr will not catch the invocation of c->bf() where c is nullptr because the this of the B sub-object inside the C object is shifted by four bytes (due to the A sub-object).
Let's try to illustrate the layout of a C object:
0: | n |
4: | m |
the numbers on the left-hand-side are offsets from the object's beginning. So, at offset 0 we have the A sub-object (with its data member n). at offset 4 we have the B sub-objects (with its data member m).
The this of the entire object, as well as the this of the A sub-object both point at offset 0. However, when we want to refer to the B sub-object (when invoking a method defined by B) the this value need to be adjusted such that it points at the beginning of the B sub-object. Hence the +4.
Note this is UB anyway.
Multiple inheritance can introduce an offset, depending on the implementation:
#include <iostream>
struct wup
{
int i;
void foo()
{
std::cout << (void*)this << std::endl;
}
};
struct dup
{
int j;
void bar()
{
std::cout << (void*)this << std::endl;
}
};
struct s : wup, dup
{
void foobar()
{
foo();
bar();
}
};
int main()
{
s* p = nullptr;
p->foobar();
}
Output on some version of clang++:
0
0x4
Live example.
Also note, as I pointed out in the comments to the OP, that this assert might not work for virtual function calls, as the vtable isn't initialized (if the compiler does a dynamic dispatch, i.e. doesn't optimize if it know the dynamic type of *p).
Here is a situation where it might happen:
struct A {
void f()
{
// this assert will probably not fail
assert(this!=nullptr);
}
};
struct B {
A a1;
A a2;
};
static void g(B *bp)
{
bp->a2.f(); // undefined behavior at this point, but many compilers will
// treat bp as a pointer to address zero and add sizeof(A) to
// the address and pass it as the this pointer to A::f().
}
int main(int,char**)
{
g(nullptr); // oops passed null!
}
This is undefined behavior for C++ in general, but with some compilers, it might have the
consistent behavior of the this pointer having some small non-zero address inside A::f().
Compilers typically implement multiple inheritance by storing the base objects sequentially in memory. If you had, e.g.:
struct bar {
int x;
int something();
};
struct baz {
int y;
int some_other_thing();
};
struct foo : public bar, public baz {};
The compiler will allocate foo and bar at the same address, and baz will be offset by sizeof(bar). So, under some implementation, it's possible that nullptr -> some_other_thing() results in a non-null this.
This example at Coliru demonstrates (assuming the result you get from the undefined behavior is the same one I did) the situation, and shows an assert(this != nullptr) failing to detect the case. (Credit to #DyP who I basically stole the example code from).
I think its not that bad a idea to put assert, for example atleast it can catch see below example
class Test{
public:
void DoSomething() {
std::cout << "Hello";
}
};
int main(int argc , char argv[]) {
Test* nullptr = 0;
nullptr->DoSomething();
}
The above example will run without error, If more complex becomes difficult to debug if that assert is absent.
I am trying to make a point that null this pointer can go unnoticed, and in complex situation becomes difficult to debug , I have faced this situation.

How does the Visual C++ compiler pass the this ptr to the called function?

I'm learning C++ using Eckel's "Thinking in C++". It states the following:
If a class contains virtual methods, a virtual function table is created for that class etc. The workings of the function table are explained roughly. (I know a vtable is not mandatory, but Visual C++ creates one.)
The calling object is passed to the called function as an argument. (This might not be true for Visual C++ (or any compiler).) I'm trying to find out how VC++ passes the calling object to the function.
To test both points in Visual C++, I've created the following class (using Visual Studio 2010, WinXP Home 32bit):
ByteExaminer.h:
#pragma once
class ByteExaminer
{
public:
short b[2];
ByteExaminer(void);
virtual void f() const;
virtual void g() const;
void bruteFG();
};
ByteExaminer.cpp:
#include "StdAfx.h"
#include "ByteExaminer.h"
using namespace std;
ByteExaminer::ByteExaminer(void)
{
b[0] = 25;
b[1] = 26;
}
void ByteExaminer::f(void) const
{
cout << "virtual f(); b[0]: " << hex << b[0] << endl;
}
void ByteExaminer::g(void) const
{
cout << "virtual g(); b[1]: " << hex << b[1] << endl;
}
void ByteExaminer::bruteFG(void)
{
int *mem = reinterpret_cast<int*>(this);
void (*fg[])(ByteExaminer*) = { (void (*)(ByteExaminer*))(*((int *)*mem)), (void (*)(ByteExaminer*))(*((int *)(*mem + 4))) };
fg[0](this);
fg[1](this);
}
The navigation through the vtable in bruteFG() works - when I call fg[0](this), f() is called. What does NOT work, however, is the passing of this to the function - meaning that this->b[0] is not printed correctly (garbage comes out instead. I'm actually lucky this doesn't produce a segfault).
So the actual output for
ByteExaminer be;
be.bruteFG();
is:
virtual f(); b[0]: 1307
virtual g(); b[1]: 0
So how should I proceed to get the correct result? How are the this pointers passed to functions in VC++?
(Nota bene: I'm NOT going to program this way seriously, ever. This is "for the lulz"; or for the learning experience. So don't try to convert me to proper C++ianity :))
Member functions in Visual Studio have a special calling convention, __thiscall, where this is passed in a special register. Which one, I don't recall, but MSDN will say. You will have to go down to assembler if you want to call a function pointer which is in a vtable.
Of course, your code exhibits massively undefined behaviour- it's only OK to alias an object using a char or unsigned char pointer, and definitely not an int pointer- even ignoring the whole vtable assumptions thing.
OK using DeadMG's hint I've found a way without using assembler:
1) Remove the ByteExaminer* arg from the functions in the fg[] array
2) Add a function void callfunc(void (*)()); to ByteExaminer:
void ByteExaminer::callfunc(void (*func)())
{
func();
}
... this apparently works because func() is the first thing to be used in callfunc, so ecx is apparently not changed before. But this is a dirty trick (as you can see in the code above, I'm always on the hunt for clean code). I'm still looking for better ways.

calling virtual functions through pointers with and without consulting the VM-table

I want to take the address of a member function of a c++ class, store it in a pointer, and call the virtual function later on.
I know some things about it, but do not now how to take the address of a certain implementation of a virtual function that is NOT the implementation of the most descendent class (the actual class of the object).
Here is some sample code:
#include <iostream>
using namespace std;
class ca
{
public:
virtual void vfunc() {cout << "a::vfunc ";}
void mfunc() {cout << "a::mfunc ";}
};
class cb : public ca
{
public:
virtual void vfunc() {cout << "b::vfunc ";}
};
extern "C" int main(int, char **)
{
void (ca:: *ptr_to_vfunc)() = &ca::vfunc;
cout << sizeof(ptr_to_vfunc) << " ";
cb b;
(b.*ptr_to_vfunc)();
ca a;
(a.*ptr_to_vfunc)();
void (ca:: *ptr_to_mfunc)() = &ca::mfunc;
cout << sizeof(ptr_to_mfunc) << " ";
(a.*ptr_to_mfunc)();
}
The output is:
12 b::vfunc a::vfunc 12 a::mfunc
I am working with win32-environment, and the size of member function pointers is 3 * 32-bits values! I did not specify an object when I took the address of the member function and yet, my call invokes the most descendant class' implementation of vfunc().
1) What is going on here? Why 12 bytes in stead of 4?
2) How can I take the address of ca::vfunc() and call it on b, like I normaly would do with b.ca::vfunc().
Ok: Its doing exactly what it it is supposed to do.
But to answer you questions:
1) What is going on here? Why 12 bytes in stead of 4?
Why not.
The standard does not specify a size.
I am not sure why you expect a normal pointer to be 4.
If the question is "why is a method pointer larger than a normal pointer?"
Because the implementation needs the extra space to hold information about the call.
2) How can I take the address of ca::vfunc() and call it on b, like I normaly would do with b.ca::vfunc().
You cant.

Virtual member functions and std::tr1::function: How does this work?

Here is a sample piece of code. Note that B is a subclass of A and both provide a unique print routine. Also notice in main that both bind calls are to &A::print, though in the latter case a reference to B is passed.
#include <iostream>
#include <tr1/functional>
struct A
{
virtual void print()
{
std::cerr << "A" << std::endl;
}
};
struct B : public A
{
virtual void print()
{
std::cerr << "B" << std::endl;
}
};
int main (int argc, char * const argv[])
{
typedef std::tr1::function<void ()> proc_t;
A a;
B b;
proc_t a_print = std::tr1::bind(&A::print, std::tr1::ref(a));
proc_t b_print = std::tr1::bind(&A::print, std::tr1::ref(b));
a_print();
b_print();
return 0;
}
Here is the output I see compiling with GCC 4.2:
A
B
I would consider this correct behavior, but I am at a loss to explain how it is working properly given that the std::tr1::functions were bound to &A::print in both cases. Can someone please enlighten me?
EDIT: Thanks for the answers. I am familiar with inheritance and polymorphic types. What I am interested in is what does &A::print mean? Is it an offset into a vtable, and that vtable changes based on the referred object (in this case, a or b?) From a more nuts-and-bolts perspective, how does this code behave correctly?
This works in the same manner as it would have worked with plain member function pointers. The following produces the same output:
int main ()
{
A a;
B b;
typedef void (A::*fp)();
fp p = &A::print;
(a.*p)(); // prints A
(b.*p)(); // prints B
}
It would have been surprising if boost/tr1/std::function did anything different since they presumably store these pointers to member functions under the hood. Oh, and of course no mention of these pointers is complete without a link to the Fast Delegates article.
Because print() is declared virtual, A is a polymorphic class. By binding to the print function pointer, you will be calling through an A pointer, much in the same way as:
A* ab = &b;
ab->print();
In the ->print call above, you would expect polymorphic behavior. Same it true in your code as well. And this is a Good Thing, if you ask me. At least, most of the time. :)

How to get every virtual function index just as the compiler does?

Is there some plugin or tool which can read a .h file (or simply modify Intellisense itself) and spit out every function and it's virtual function table index? There's a pattern which I have yet to figure out having to do with polymorphism, and it gets 5x harder when you start to have 5 classes or more deriving from each other. No matter what, though, the MSVC++ compiler always spits out the correct virtual function table index when it compiles the virtual function call from C++ to Assembly. There has to be a better way to get that index without loading, break-pointing, reading the offset, and rewriting the code, right?
Thanks!
Use the hidden Microsoft C/C++ compiler option "/d1 reportAllClassLayout". This will print out the memory layout and vtables of all your classes.
You could try writing a hack that determines it for you - this is implementation defined, but you can usually find a pointer to the virtual function table at the beginning of class memory. If you follow to this table there'll be a list of function pointers in memory (but you won't know how many). However, by searching for the functions you know about in the table of function pointers, you could identify its index.
In MSVC you can't browse at runtime the vtable and compare equality with a given member function pointer as they are not the same. One is the real pointer, the other is a pointer which indirects to the real one.
However with this compiler you can do it at runtime with another hack I discovered.
Create a class (named IndexFinder for example) inside which you declare as many instance methods as the max virtual methods you can have in a class. Each of these method must return an unique integer value starting from 0 to your max.
Create a fake virtual method table and store your method pointers so that the integer they return match the index in which you store them (the method which returns 0 will be the first in your fake vtable).
When you want to find the index of any virtual method you have to do some dirty cast of the method member pointer to a IndexFinder method pointer.
The principle is easy : for virtual methods the compiler will generate code which indirects to the real method using the vtable with the good index. As you replaced the compiler generated vtable with a fake one, it will jump to yours and not to the supposed one. As your method returns the index inside which it's stored, you just have to get the return and you have your index.
Here is a code which is more explicit (I repeat it's a compiler dependant hack, the ones who don't like that, don't read it ^^). But I tried it, it works perfectly as it's just a redirection hack (I'm looking for a trick with GCC, but i haven't find it yet).
It's possible that it depends on the call convention, i haven't tried it in all cases for now.
One advantage of this trick is that you don't need to construct an instance of your class to find the index of one of it's virtual methods.
// In the header .h
class IndexFinder
{
typedef int (IndexFinder::*method_pointer)();
public:
template<typename _MethodPtr>
int getIndexOf(_MethodPtr ptr) {
return (reinterpret_cast<IndexFinder*>(&fake_vtable_ptr)->**((IndexFinder::method_pointer*)(&ptr)))()
}
protected:
int method0() { return 0; }
int method1() { return 1; }
int method2() { return 2; }
int method3() { return 3; }
protected:
typedef method_pointer fake_vtable_t [4];
static fake_vtable_t fake_vtable;
void* fake_vtable_ptr;
};
// In the cpp file
IndexFinder::fake_vtable_t IndexFinder::fake_vtable = {
&IndexFinder::method0 ,
&IndexFinder::method1 ,
&IndexFinder::method2 ,
&IndexFinder::method3
};
void* IndexFinder::fake_vtable_ptr = &IndexFinder::fake_vtable;
// to use it :
int index = IndexFinder::getIndexOf(&YourClass::yourVirtualMethod);
You can use Developer Command Prompt
cl /d1 reportSingleClassLayoutXXX filename
XXX -> class name
Example
class CBase
{
public:
CBase() {
};
virtual void Walk() { cout << "CBase:Walk" << endl; }
virtual void Jump() { cout << "CBase:Jump" << endl; }
void Run(int speed) { cout << "CBase:Run:" << "Speed=" << speed << endl; }
};
class CDerivedA : public CBase
{
public:
CDerivedA() {
};
void Walk() { cout << "CDerivedA:Walk" << endl; }
void Jump() { cout << "CDerivedA:Jump" << endl; }
void Run(int speed) { cout << "CDerivedA:Run" << "Speed=" << speed << endl; }
};
Layout
D:\nicolas>cl /d1 reportSingleClassLayoutCDerivedA nicolas.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 19.10.25019 for x86
Copyright (C) Microsoft Corporation. All rights reserved.
class CDerivedA size(4):
+---
0 | +--- (base class CBase)
0 | | {vfptr}
| +---
+---
CDerivedA::$vftable#:
| &CDerivedA_meta
| 0
0 | &CDerivedA::Walk
1 | &CDerivedA::Jump
CDerivedA::Walk this adjustor: 0
CDerivedA::Jump this adjustor: 0