I know this has been discussed a few times, but my situation is a bit different.
I have a third-party dll exporting some classes. Unfortunately, the header file is not available.
It is still possible to call exported functions. But I cannot get around passing the right 'this' pointer (which is passed in RCX register).
First I use dumpbin /exports to extract the function names (names are changed as the third-party library and function names are confidential).
4873 1308 0018B380 ?GetId#ThirdPartyClass#ThirdPartyNamespace##QEBAJXZ = ??GetId#ThirdPartyClass#ThirdPartyNamespace##QEBAJXZ (public: long __cdecl ThirdPartyNamespace::ThirdPartyClass::GetId(void)const )
Now, the API allows me to register my callback that receives a pointer to ThirdPartyNamespace::ThirdPartyClass (there is only forward declaration of ThirdPartyClass).
Here how I am trying to call ThirdPartyNamespace::ThirdPartyClass::GetId():
long (ThirdPartyNamespace::ThirdPartyClass::*_pFnGetId)() const;
HMODULE hModule = GetModuleHandle("ThirdPartyDLL.dll");
*(FARPROC*)&_pFnGetId= GetProcAddress(hModule, "?GetId#ThirdPartyClass#ThirdPartyNamespace##QEBAJXZ");
long id = (ptr->*_pFnGetId)();
Everything looks fine (i.e. if I step in - I get indeed inside ThirdPartyClass::GetId method. But the this pointer is not good. While the ptr is good and if in debugger I manually change rcx to the ptr - it works fine. But compiler does not pass ptr for some reason. Here is disassembly:
long id = (ptr->*_pFnGetId)();
000000005C882362 movsxd rax,dword ptr [rdi+30h]
000000005C882366 test eax,eax
000000005C882368 jne MyClass::MyCallback+223h (05C882373h)
000000005C88236A movsxd rcx,dword ptr [rdi+28h]
000000005C88236E add rcx,rsi
000000005C882371 jmp MyClass::MyCallback+240h (05C882390h)
000000005C882373 movsxd r8,dword ptr [rdi+2Ch]
000000005C882377 mov rcx,rax
000000005C88237A mov rax,qword ptr [r8+rsi]
000000005C88237E movsxd rdx,dword ptr [rax+rcx]
000000005C882382 movsxd rcx,dword ptr [rdi+28h]
000000005C882386 lea rax,[r8+rdx]
000000005C88238A add rcx,rax
000000005C88238D add rcx,rsi
000000005C882390 call qword ptr [rdi+20h]
000000005C882393 mov ebp,eax
Before executing these commands, rsi contains the pointer to the object of ThirdPartyClass (i.e. ptr), but instead of passing it in rcx directly, some arithmetic is performed on it and as a result, this pointer gets completely wrong.
some traces which I don't understand why compiler is doing it as it end up calling non-virtual function ThirdPartyClass::GetId():
000000005C88237A mov rax,qword ptr [r8+rsi]
R8 0000000000000000
RSI 000000004C691AA0 // good pointer to ThirdPartyClass object
RAX 0000000008E87728 // this gets pointer to virtual functions table of ThirdPartyClass
000000005C88237E movsxd rdx,dword ptr [rax+rcx]
RAX 0000000008E87728
RCX FFFFFFFFFFFFFFFF
RDX FFFFFFFFC0F3C600
000000005C882382 movsxd rcx,dword ptr [rdi+28h]
RCX 0000000000000000
RDI 000000005C9BE690
000000005C882386 lea rax,[r8+rdx]
RAX FFFFFFFFC0F3C600
RDX FFFFFFFFC0F3C600
R8 0000000000000000
000000005C88238A add rcx,rax
RAX FFFFFFFFC0F3C600
RCX FFFFFFFFC0F3C600
000000005C88238D add rcx,rsi
RCX 000000000D5CE0A0
RSI 000000004C691AA0
000000005C882390 call qword ptr [rdi+20h]
In my view, it should be as simple as
long id = (ptr->*_pFnGetId)();
mov rcx,rsi
call qword ptr [rdi+20h]
mov ebp,eax
And if I set rcx equal to rsi before the call qword ptr [rdi+20h] it returns me expected value.
Am I doing something completely wrong?
Thanks in advance.
Ok, I found a solution, by incident (as I already used similar approach and it worked in slightly different situation.
The solution is to trick the compiler by defining a fake class and calling member method by pointer, but pretending that it is a pointer to the known (to compiler) class.
Perhaps, it does not matter, but I know that ThirdPartyNamespace::ThirdPartyClass has virtual functions, so I declare fake class with virtual function as well.
class FakeCall
{
private:
FakeCall(){}
virtual ~FakeCall(){}
};
The rest as in the initial code except once small thing, instead of calling ptr->*_pFnGetId (where ptr is pointer to unknown, forward declared class ThirdPartyNamespace::ThirdPartyClass), I am pretending I am calling member method in my FakeCall class:
FakeCall * fake = (FakeCall*)ptr;
long sico = (fake->*_pFnGetId)();
Disassembly looks exactly as expected:
long sico = (fake->*_pFnGetSico)();
000000005A612096 mov rcx,rax
000000005A612099 call qword ptr [r12+20h]
000000005A61209E mov esi,eax
And it works perfectly!
Some observations:
The member method pointer, as I thought initially, nothing more than a normal function pointer.
Microsoft compiler (at least VS2008) goes crazy if calling member method for not defined class (i.e. only forward declaration of the name).
Related
In some code i have seen the following:
(&object)->something.
Is there any advantage to object.something ?
Does the compiler somehow optimize such code, or is it faster in any way?
If operator& is not overloaded it's essentially the same https://godbolt.org/g/iPTjRY:
auto v_1 = f_1.get();
auto v_2 = (&f_1)->get();
resolved to pretty much the same:
lea rax, [rbp-12] ; load object address
mov rdi, rax ; move object address into rdi, not sure why not just: 'lea rdi, [rbp-12]'
call Foo::get() const ; invoke the subroutine
mov DWORD PTR [rbp-4], eax ; save the result at [rbp-4]
(already with no optimizations they are the same; with optmizations turned on... the entire calls get discarded, so that's left for the curious reader)
Assuming that code is located inside if block, what are differences between creating object in a free store and doing only one call on it:
auto a = aFactory.createA();
int result = a->foo(5);
and making call directly on returned pointer?
int result = aFactory.createA()->foo(5);
Is there any difference in performance? Which way is better?
#include <iostream>
#include <memory>
class A
{
public:
int foo(int a){return a+3;}
};
class AFactory
{
public:
std::unique_ptr<A> createA(){return std::make_unique<A>();}
};
int main()
{
AFactory aFactory;
bool condition = true;
if(condition)
{
auto a = aFactory.createA();
int result = a->foo(5);
}
}
Look here. There is no difference in code generated for both versions even with optimisations disabled.
Using gcc7.1 with -std=c++1z -O0
auto a = aFactory.createA();
int result = a->foo(5);
is compiled to:
lea rax, [rbp-24]
lea rdx, [rbp-9]
mov rsi, rdx
mov rdi, rax
call AFactory::createA()
lea rax, [rbp-24]
mov rdi, rax
call std::unique_ptr<A, std::default_delete<A> >::operator->() const
mov esi, 5
mov rdi, rax
call A::foo(int)
mov DWORD PTR [rbp-8], eax
lea rax, [rbp-24]
mov rdi, rax
call std::unique_ptr<A, std::default_delete<A> >::~unique_ptr()
and int result = aFactory.createA()->foo(5); to:
lea rax, [rbp-16]
lea rdx, [rbp-17]
mov rsi, rdx
mov rdi, rax
call AFactory::createA()
lea rax, [rbp-16]
mov rdi, rax
call std::unique_ptr<A, std::default_delete<A> >::operator->() const
mov esi, 5
mov rdi, rax
call A::foo(int)
mov DWORD PTR [rbp-8], eax
lea rax, [rbp-16]
mov rdi, rax
call std::unique_ptr<A, std::default_delete<A> >::~unique_ptr()
So they are pretty much identical.
This outcome is understandable when you realise the only difference between the two versions is that in the first one we assign the name to our object, while in the second we work with an unnamed one. Other then that, they are both created on heap and used the same way. And since variable name means nothing to the compiler - it is only relevant for code-readers - it treats the two as if they were identical.
In your simple case it will not make a difference, because the (main) function ends right after creating and using a.
If some more lines of code would follow, the destruction of the a object would happen at the end of the if block in main, while in the one line case it becomes destructed at the end of that single line. However, it would be bad design if the destructor of a more sophisticated class A would make a difference on that.
Due to compiler optimizations performance questions should always be answered by testing with a profiler on the concrete code.
There is a class SomeClass which holds some data and methods that operates on this data. And it must be created with some arguments like:
SomeClass(int some_val, float another_val);
There is another class, say Manager, which includes SomeClass, and heavily uses its methods.
So, what would be better in terms of performance (data locality, cache hits, etc.), declare object of SomeClass as member of Manager and use member initialization in Manager's constructor or declare object of SomeClass as unique_ptr?
class Manager
{
public:
Manager() : some(5, 3.0f) {}
private:
SomeClass some;
};
or
class Manager
{
public:
Manager();
private:
std::unique_ptr<SomeClass> some;
}
Short answer
Most likely, there is no difference in runtime efficiency of accessing your subobject. But using pointer can be slower for several reasons (see details below).
Moreover, there are several other things you should remember:
When using pointer, you usually have to allocate/deallocate memory for subobject separately, which takes some time (quite a lot if you do it much).
When using pointer, you can cheaply move your subobject without copying.
Speaking of compile times, pointer is better than plain member. With plain member, you cannot remove dependency of Manager declaration on SomeClass declaration. With pointers, you can do it with forward declaration. Less dependencies may result is less build times.
Details
I'd like to provide more details about performance of subobject accesses. I think that using pointer can be slower than using plain member for several reasons:
Data locality (and cache performance) is likely to be better with plain member. You usually access data of Manager and SomeClass together, and plain member is guaranteed to be near other data, while heap allocations may place object and subobject far from each other.
Using pointer means one more level of indirection. To get address of a plain member, you can simply add a compile-time constant offset fo object address (which is often merged with other assembly instruction). When using pointer, you have to additionally read a word from the member pointer to get actual pointer to subobject. See Q1 and Q2 for more details.
Aliasing is perhaps the most important issue. If you are using plain member, then compiler can assume that: your subobject lies fully within your object in memory, and it does not overlap with other members of your object. When using pointer, compiler often cannot assume anything like this: you subobject may overlap with your object and its members. As a result, compiler has to generate more useless load/store operations, because it thinks that some values may change.
Here is an example for the last issue (full code is here):
struct IntValue {
int x;
IntValue(int x) : x(x) {}
};
class MyClass_Ptr {
unique_ptr<IntValue> a, b, c;
public:
void Compute() {
a->x += b->x + c->x;
b->x += a->x + c->x;
c->x += a->x + b->x;
}
};
Clearly, it is stupid to store subobjects a, b, c by pointers. I've measured time spent in one billion calls of Compute method for a single object. Here are results with different configurations:
2.3 sec: plain member (MinGW 5.1.0)
2.0 sec: plain member (MSVC 2013)
4.3 sec: unique_ptr (MinGW 5.1.0)
9.3 sec: unique_ptr (MSVC 2013)
When looking at the generated assembly for innermost loop in each case, it is easy to understand why the times are so different:
;;; plain member (GCC)
lea edx, [rcx+rax] ; well-optimized code: only additions on registers
add r8d, edx ; all 6 additions present (no CSE optimization)
lea edx, [r8+rax] ; ('lea' instruction is also addition BTW)
add ecx, edx
lea edx, [r8+rcx]
add eax, edx
sub r9d, 1
jne .L3
;;; plain member (MSVC)
add ecx, r8d ; well-optimized code: only additions on registers
add edx, ecx ; 5 additions instead of 6 due to a common subexpression eliminated
add ecx, edx
add r8d, edx
add r8d, ecx
dec r9
jne SHORT $LL6#main
;;; unique_ptr (GCC)
add eax, DWORD PTR [rcx] ; slow code: a lot of memory accesses
add eax, DWORD PTR [rdx] ; each addition loads value from memory
mov DWORD PTR [rdx], eax ; each sum is stored to memory
add eax, DWORD PTR [r8] ; compiler is afraid that some values may be at same address
add eax, DWORD PTR [rcx]
mov DWORD PTR [rcx], eax
add eax, DWORD PTR [rdx]
add eax, DWORD PTR [r8]
sub r9d, 1
mov DWORD PTR [r8], eax
jne .L4
;;; unique_ptr (MSVC)
mov r9, QWORD PTR [rbx] ; awful code: 15 loads, 3 stores
mov rcx, QWORD PTR [rbx+8] ; compiler thinks that values may share
mov rdx, QWORD PTR [rbx+16] ; same address with pointers to values!
mov r8d, DWORD PTR [rcx]
add r8d, DWORD PTR [rdx]
add DWORD PTR [r9], r8d
mov r8, QWORD PTR [rbx+8]
mov rcx, QWORD PTR [rbx] ; load value of 'a' pointer from memory
mov rax, QWORD PTR [rbx+16]
mov edx, DWORD PTR [rcx] ; load value of 'a->x' from memory
add edx, DWORD PTR [rax] ; add the 'c->x' value
add DWORD PTR [r8], edx ; add sum 'a->x + c->x' to 'b->x'
mov r9, QWORD PTR [rbx+16]
mov rax, QWORD PTR [rbx] ; load value of 'a' pointer again =)
mov rdx, QWORD PTR [rbx+8]
mov r8d, DWORD PTR [rax]
add r8d, DWORD PTR [rdx]
add DWORD PTR [r9], r8d
dec rsi
jne SHORT $LL3#main
I have a weird problem when I create a release build with Visual Studio 2008. I was wondering if one of you could help me understand what is going on.
Description:
I have a class member function which returns a pointer to a structure stored in the class:
const MyStruct * Myclass::getPointer() {
return mystruct_variable; // this is properly initialyzed
}
One another point worth pointing out is that this class/method is in a dll, and I export it to use in a separate executable project. When I make a release build and try to use the above mentioned method, the run crashes depending on if the getPointer() method is inlined (i.e. placed in the header file part of the class) or not (placed in the cpp file).
The usage is:
const MyStruct * cf = myclassObj.getPointer();
int n = cf->size_t;
std::cout<<n<<std::endl;
When MyClass::getPointer() is inlined in the header the assembly looks like:
const MyStruct * cf = myclassObj.getPointer();
012514A8 mov esi,dword ptr [ebx+794h]
int n =cf->size_t;
012514AE mov eax,dword ptr [esi+20h]
std::cout<<n<<std::endl;
012514B1 mov ecx,dword ptr [__imp_std::endl (1252038h)]
012514B7 push ecx
012514B8 mov ecx,dword ptr [__imp_std::cout (125203Ch)]
012514BE push eax
012514BF call dword ptr [__imp_std::basic_ostream<char,std::char_traits<char> >::operator<< (1252048h)]
012514C5 mov ecx,eax
012514C7 call dword ptr [__imp_std::basic_ostream<char,std::char_traits<char> >::operator<< (1252044h)]
The same code when the class method for getPointer() is not inlined, and placed in the corresponding cpp file gives:
const MyStruct * cf = myclassObj.getPointer();
00DA14A8 mov ecx,ebx
00DA14AA call dword ptr [__imp_MyClass::getPointer(0DA2104h)]
int n =cf->size_t;
std::cout<<n<<std::endl;
00DA14B0 mov ecx,dword ptr [__imp_std::endl (0DA2038h)]
00DA14B6 mov esi,eax
00DA14B8 mov eax,dword ptr [esi+20h]
00DA14BB push ecx
00DA14BC mov ecx,dword ptr [__imp_std::cout (0DA203Ch)]
00DA14C2 push eax
00DA14C3 call dword ptr [__imp_std::basic_ostream<char,std::char_traits<char> >::operator<< (0DA2048h)]
00DA14C9 mov ecx,eax
00DA14CB call dword ptr [__imp_std::basic_ostream<char,std::char_traits<char> >::operator<< (0DA2044h)]
Any thoughts why the two cases have different assembly set? Am I doing something wrong?
Thanks!
If you link against a C++ DLL you must make sure all the compiler flags are exactly the same. Otherwise the size of structures, virtual tables etc. might be different and the code fails with an invalid access to memory. Inlining of course overcomes this as the code is inside the exe file, not in the dll, and thus compiled with the correct flags.
Simply put - for a release build use release DLL, for a debug build use debug DLL.
If you've been programming for a while then you probably noticed something completely impossible occurs every now and then for which you are convinced there is no possible explanation ("IT'S A COMPILER BUG!!"). After you find out what it was caused by though you are like "oooohhh".
Well, it just happened to me :(
Here AuthDb is NOT NULL but a valid pointer:
SingleResult sr(AuthDb, format("SELECT Id, Access, Flags, SessionKey, RealmSplitPreference FROM accounts WHERE Name = '%s'") % Escaped(account_name));
Here it mysteriously becomes NULL:
struct SingleResult : public BaseResult
{
SingleResult(Database *db, const boost::format& query) { _ExecuteQuery(db, query.str()); }
}
Notice that it's the immediate next call. It can be explained much better with two screenshots:
http://img187.imageshack.us/img187/5757/ss1zm.png
http://img513.imageshack.us/img513/5610/ss2b.png
EDIT: AuthDb is a global variable. It itself keeps pointing to the right thing; but the copy of the ptr Database *db points at NULL.
ASM code (unfortunately I don't even know how to read it :/)
Of the first screenshot
01214E06 mov eax,dword ptr [ebp-328h]
01214E0C push eax
01214E0D push offset string "SELECT Id, Access, Flags, Sessio"... (13C6278h)
01214E12 lea ecx,[ebp-150h]
01214E18 call boost::basic_format<char,std::char_traits<char>,std::allocator<char> >::basic_format<char,std::char_traits<char>,std::allocator<char> > (11A3260h)
01214E1D mov dword ptr [ebp-32Ch],eax
01214E23 mov ecx,dword ptr [ebp-32Ch]
01214E29 mov dword ptr [ebp-330h],ecx
01214E2F mov byte ptr [ebp-4],2
01214E33 mov ecx,dword ptr [ebp-330h]
01214E39 call boost::basic_format<char,std::char_traits<char>,std::allocator<char> >::operator%<Snow::Escaped> (11A3E18h)
01214E3E push eax
01214E3F mov edx,dword ptr [__tls_index (144EC40h)]
01214E45 mov eax,dword ptr fs:[0000002Ch]
01214E4B mov ecx,dword ptr [eax+edx*4]
01214E4E mov edx,dword ptr [ecx+12A3Ch]
01214E54 push edx
01214E55 lea ecx,[sr]
01214E58 call Snow::SingleResult::SingleResult (11A27D4h)
01214E5D mov byte ptr [ebp-4],4 // VS GREEN ARROW IS HERE
01214E61 lea ecx,[ebp-150h]
01214E67 call boost::basic_format<char,std::char_traits<char>,std::allocator<char> >::~basic_format<char,std::char_traits<char>,std::allocator<char> > (11A1DBBh)
01214E6C mov byte ptr [ebp-4],5
01214E70 lea ecx,[ebp-170h]
01214E76 call Snow::Escaped::~Escaped (11A42D2h)
const bool account_found = !sr.Error();
01214E7B lea ecx,[sr]
01214E7E call Snow::BaseResult::Error (11A2964h)
01214E83 movzx eax,al
01214E86 test eax,eax
01214E88 sete cl
01214E8B mov byte ptr [account_found],cl
if (!account_found) {
01214E8E movzx edx,byte ptr [account_found]
01214E92 test edx,edx
01214E94 jne AuthSession+1C0h (1214F10h)
client.Kill(format("%s: Attempted to login with non existant account `%s'") % client % account_name, true);
Second screenshot
011A8E7D mov dword ptr [ebp-10h],ecx
011A8E80 mov ecx,dword ptr [this]
011A8E83 call Snow::BaseResult::BaseResult (11A31D9h)
011A8E88 mov dword ptr [ebp-4],0
011A8E8F lea eax,[ebp-30h]
011A8E92 push eax
011A8E93 mov ecx,dword ptr [query]
011A8E96 call boost::basic_format<char,std::char_traits<char>,std::allocator<char> >::str (11A1E01h)
011A8E9B mov dword ptr [ebp-34h],eax
011A8E9E mov ecx,dword ptr [ebp-34h]
011A8EA1 mov dword ptr [ebp-38h],ecx
011A8EA4 mov byte ptr [ebp-4],1
011A8EA8 mov edx,dword ptr [ebp-38h]
011A8EAB push edx
011A8EAC mov eax,dword ptr [db]
011A8EAF push eax
011A8EB0 mov ecx,dword ptr [this]
011A8EB3 call Snow::SingleResult::_ExecuteQuery (124F380h)
011A8EB8 mov byte ptr [ebp-4],0 // VS GREEN ARROW HERE
011A8EBC lea ecx,[ebp-30h]
011A8EBF call std::basic_string<char,std::char_traits<char>,std::allocator<char> >::~basic_string<char,std::char_traits<char>,std::allocator<char> > (11A2C02h)
011A8EC4 mov dword ptr [ebp-4],0FFFFFFFFh
011A8ECB mov eax,dword ptr [this]
011A8ECE mov ecx,dword ptr [ebp-0Ch]
011A8ED1 mov dword ptr fs:[0],ecx
011A8ED8 pop edi
011A8ED9 add esp,38h
011A8EDC cmp ebp,esp
011A8EDE call _RTC_CheckEsp (12B4450h)
011A8EE3 mov esp,ebp
011A8EE5 pop ebp
011A8EE6 ret 8
UPDATE
Following the suggestion of peterchen, I added ASSERT(AuthDb); here:
ASSERT(AuthDb);
SingleResult sr(AuthDb, format("SELECT Id, Access, Flags, SessionKey, RealmSplitPreference FROM accounts WHERE Name = '%s'") % Escaped(account_name));
And it failed O.o And yet the debugger keeps insisting that it's not NULL.. It's not shadowed by a local
UPDATE2*
cout << AuthDb; is 0 in there even if the debugger says it's not
FOUND THE PROBLEM
Database *AuthDb = NULL, *GameDb = NULL; in a .cpp
extern thread Database *AuthDb, *GameDb; in a .h
The variable was marked thread (TLS - Thread local storage) in the header, but not TLS in the definition...
Countless hours wasted on this super stupid mistake, no warnings or hints or anything from the compiler that I feel like killing right now. :( Oh well, like I said for each impossible behavior there is a solution that once known seems obvious :)
Thanks to everyone who helped, I was really desperate!
Is AuthDB a thread-local variable?
Maybe the debugger isn't handling it correctly. What if you ASSERT(AuthDB) before the constructor is called?
UPDATE: If it is thread-local, it simply hasn't been initialized in this thread.
One possibility is that in the second screenshot, you have the debugger stopped at the very beginning of the function, before the stack has been manipulated, and so the variable locations aren't correct. You might also be after the end of the function, where the stack has already been torn down. I've seen that sort of thing before.
Expand that function to be a multiline function, so that it looks like this:
struct SingleResult : public BaseResult
{
SingleResult(Database *db, const boost::format& query)
{
_ExecuteQuery(db, query.str());
}
}
... and see if it still shows db as null when you have the debugger stopped on the _ExecuteQuery line.
Do you have a local AuthDB that is null and hides your global one?
(I'd expect the debugger in this case to correctly show you the local one... but with VS quality decay, I'd not rely on that)
I'd change the code to:
_ASSERTE(AuthDb);
SingleResult sr(AuthDb, format(...));
....
struct SingleResult : public BaseResult
{
SingleResult(Database *db, const boost::format& query)
{
_ASSERTE(db);
_ExecuteQuery(db, query.str());
}
}
And follow the disassembly in the debugger.
Well I'm not sure what classes/functions you're using here, but from a quick glance, shouldn't it be:
SingleResult sr(AuthDb, format("SELECT Id, Access, Flags, SessionKey, RealmSplitPreference FROM accounts WHERE Name = '%s'", Escaped(account_name)));
instead of:
SingleResult sr(AuthDb, format("SELECT Id, Access, Flags, SessionKey, RealmSplitPreference FROM accounts WHERE Name = '%s'") % Escaped(account_name));
It seems to me you're putting a modulus of the of Escaped(account_name) rather than passing that as an argument to format. However, I could just be confused.
Mike
I have no guess what is going on, but if I were debugging this I would be looking at the assembly language to see what is happening. You may need to get a better understanding of your platforms calling convention (i.e. how are arguments passed, on the stack, in registers, etc.) to troubleshoot this.
There is likely a bug somewhere else in your program. I suggest you find the problem by looking elsewhere in your code.
Another possibility--you might be looking at a memory overwrite due to a wild pointer somewhere.
Assuming your debugger supports it when you hit the first line set a memory-write breakpoint on AuthDb.