Loading an EXE as a DLL, local vftable - c++

I have an exe named test.exe which is usually used as a stand-alone application. I want to use this exe as a module (a dll) inside another application, app.exe.
The code in test.exe does something really simple like:
void doTest()
{
MyClass *inst = new MyClass();
inst->someMethod();
}
Where someMethod() is virtual and MyClass has a virtual d'tor.
doTest() is exported from test.exe and thus a lib called test.lib is created
app.exe is linked with this lib to statically load test.exe when it starts.
When I'm running test.exe stand-alone it runs just fine but when I'm running it loaded from within app.exe it crashes.
Stepping into the code with the debugger revealed that the crash is in the call to the virtual method. It turns out that the vftable somehow goes bad.
After some investigations it turns out that when the code inside the constructor of MyClass is running , the vftable is one thing but when the call to new returns it is replace with something else called a "local vftable". I found this obscure discussion about why this is.
After about a day of debugging it occurred to me that the pointers in this "local vftable" are the same in both cases, when test.exe is stand alone and whenit is loaded as a module. This can't be right because test.exe is loaded into a different address...
To test this theory I changed the loading address in the linker options to the one where test.exe is loaded when it is in app.exe and now, lo and behold, everything works.
Obviously, this is not a permanent solution because next time this randomly selected address may be occupied and the same problem will occur again.
So my question: Why is this "local vftable" tied to the static loading address of the exe? is loading an exe as a module a bad thing? why does the exe assume it is loaded to its static address?
Just for context: this is all done with MSVC 2008, Windows XP x64.

VC++ strips out reloc info from .exes by default because normally they don't need to be relocatable.
You can force it to retain the reloc info with /fixed:no. See: http://msdn.microsoft.com/en-us/library/w368ysh2.aspx

The workaround I ended up using is to simply add a compile configuration and compile the exe as a real dll instead of forcing it to act like one.
using /fixed:no didn't solve the problem for some reason.
Another difference I between exes and DLLs is that the entry point is different. a DLL's entry point is DllMain where as an exe has its entry point in the CRT which eventually calls main() or WinMain().

Related

cant load dll that contains call to CreateProcessAsUserW

i have a program that loads dlls, all dlls have only one simple c function "run". one dll contains call to CreateProcessAsUserW and when i try to load it, LoadLibrary reports error 127 "ERROR_PROC_NOT_FOUND". when i comment it out, dll loads normally and is callable. i was running dependency walker on this dll and CreateProcessAsUserW is marked red with error:
Error: At least one module has an unresolved import due to a missing export function in an implicitly dependent module.
this problem does not occur when using CreateProcessAsUserA, only when using CreateProcessAsUserW.
my setup:
xp sp3 updated to max
mingw64, gcc 4.9.1
i also have static libs and static runtime turned off
whats the source of my problem? how can i fix this so CreateProcessAsUserW will work?

Calling to a pointer from a c++ thread

I'm trying to call from a thread to a pointer.
Here is my code:
myDll.dll c++ :
long cbAddrAsync;
void _asyncer(void* data)
{
typedef void (__stdcall *FUNCPTR)();
FUNCPTR vbFunc;
vbFunc = (FUNCPTR)cbAddrAsync;
vbFunc();
}
extern "C" __declspec(dllexport) void async(long addr)
{
cbAddrAsync = addr;
HANDLE hHandle = (HANDLE)_beginthread(_asyncer,0,NULL);
}
calling to this extern with vb6:
In Module1:
Declare Sub async Lib "myDll.dll" (ByVal addr As Long)
Sub onAsync()
MsgBox "ASYNC"
End Sub
In Form1:
Private Sub Command_Click()
Call async(AddressOf Module1.onAsync)
End Sub
when i click at command button , The problem occurs in
dll:
Unhandled exception at 0x734f9232 in Project1.exe: 0xC0000005: Access violation reading location 0x00000076.
in vbFunc();
How can I fix it?
Thanks.
0x00000076 is an invalid memory address. All addresses under 0x0000ffff are invalid. This catches a whole class of errors (failed memory allocations or object creation) along the lines of 0x0 and 0x0 + 2 (or 0x0 + 0x76) etc. The errors are errors that programmers are supposed to check for but often don't.
My guess is you are accessing the 0x76th byte of a structure that wasn't initialised.
VB6 is designed to be a COM server not a dynamic load dll.
AddressOf has rules because you are actually calling VB6 runtime not your program.
You can also start in a debugger.
windbg or ntsd (ntsd is a console program and maybe installed). Both are also from Debugging Tools For Windows.
Download and install Debugging Tools for Windows
http://msdn.microsoft.com/en-us/windows/hardware/hh852363
Install the Windows SDK but just choose the debugging tools.
Create a folder called Symbols in C:\
Start Windbg. File menu - Symbol File Path and enter
srv*C:\symbols*http://msdl.microsoft.com/download/symbols
then
windbg -o -g -G c:\windows\system32\cmd.exe /k batfile.bat
You can press F12 to stop it and kb will show the call stack (g continues the program). If there's errors it will also stop and show them.
Type lm to list loaded modules, x *!* to list the symbols and bp symbolname to set a breakpoint
If programming in VB6 then this environmental variable link=/pdb:none stores the symbols in the dll rather than seperate files. Make sure you compile the program with No Optimisations and tick the box for Create Symbolic Debug Info. Both on the Compile tab in the Project's Properties.
Also CoClassSyms (microsoft.com/msj/0399/hood/hood0399.aspx) can make symbols from type libraries.

QString::toStdString() crashes on std::string destructor

I've been debugging this for 2 hours now, and it boils down to this. If I call QString::toStdString
QString s = "testtesttesttesttesttest";
const std::string &temp = s.toStdString();
the program later crashes on std::string destructor
__CLR_OR_THIS_CALL ~basic_string()
{ // destroy the string
_Tidy(true); // <---- It crashes on this line.
}
I thought it was memory corruption at first, but this happens even if main() contains only those 2 lines. Does anyone know why this happens, and also how can I fix it?
My Qt version is 4.8.1.
Your Qt DLLs need to be compiled with STL support and exactly the same C-Runtime Library as your code. It looks as though you are using two different CRTs at the same time, which would destroy the objects created on one heap by Qt into the heap used by your program.
Check the DLL Usage with the Dependency Walker!
Most probable reason could be that your Runtime Library is "Multi-threaded (/MT)" and you need to change it to "Multi-threaded DLL (/MD)" (if you are on the release version)
If you are on the debug version change from "Multi-threaded Debug (/MTd)" to "Multi-threaded Debug DLL (/MDd)"
If you have an odd compilation of Qt, the solution should be the opposite.
You will find that on "Configuration properties->C/C++->Code Generation->Runtime Library"
I tried tackling the problem a different way. I created a new project from Visual Studio, and the test code didn't crash there. Upon examining the differences between the *.vcproj files with WinMerge, I found that the crash is caused by some custom changes in the project concerning - you guessed it - the runtime libraries. This is the patch created by WinMerge with the minimum differences that cause the crash to be reproduced:
112c112
< RuntimeLibrary="3"
---
> RuntimeLibrary="1"
126a127,128
> LinkLibraryDependencies="true"
> UseLibraryDependencyInputs="false"
127a130,131
> IgnoreAllDefaultLibraries="false"
> IgnoreDefaultLibraryNames="msvcrtd.lib"

Dynamic Linking to visa32.dll causes crash

I am trying to communicate with an oscilloscope over USB with NI-VISA 5.1.1. My 32-bit application is written in the Qt framework using C++ and the MinGW compiler (standard Qt 4.7 SDK).
My application has no problems communicating with the device when I statically link to visa32.lib. My dynamically linked code also runs fine on a Windows 7 64-bit platform, but crashes on two different computers running on Windows XP SP3 (32 bit). I have traced the line of code that causes the crash (it is the first call to viWrite).
The issue seems to be related to setup of global variables in the DLL itself. This might explain why it works in Windows 7 since in that case the 32-bit DLL file is only a stub which calls a 64-bit VISA DLL file.
This is how I dynamically link to the exported functions:
typedef long (*tviRead)(ViSession, ViPBuf, ViUInt32, ViPUInt32);
typedef long (*tviWrite)(ViSession, ViBuf, ViUInt32, ViPUInt32);
QLibrary visa32("visa32"); // Qt wrapper for LoadLibrary
if (!visa32.load()) throw "Unable to load visa32.dll";
pviWrite = (tviWrite) visa32.resolve("viWrite");
pviRead = (tviRead) visa32.resolve("viRead");
If I call the following (where the vi prefixed function are statically linked and pvi ones are runtime linked),
status = viOpenDefaultRM(&rm);
status = viFindRsrc(rm, query, &list, &itemCnt, desc);
status = viOpen(rm, desc, VI_NULL, VI_NULL, &vi);
status = pviWrite(vi, (ViByte*) idn, 5, &retCnt); // Crash right here
status = pviRead(vi, (ViByte*) id, sizeof(id), &retCnt);
everything works perfectly. However changing any of the three setup function to runtime linked causes a segmentation fault when calling pviWrite. There aren't AFAIK any other functions exported by the DLL to "setup global variables", in fact I can't see DllMain exported using the Dependency Walker tool. Has anyone ever runtime linked to visa32.dll successfully? I can't find any examples on the Internet.
I suspect you have the following problem:
http://msdn.microsoft.com/en-us/library/ms684179(v=vs.85).aspx
Scroll down to the remarks section, and look for "Visual C++": if the DLL contains _declspec-style thread local storage you can only load it dynamically in Vista or later.
I'm trying to achieve the same thing but apparently, for XP, we are simply out of luck...

VS2008 debugger and kernel32.dll

I've been just debugging a process (in C++/windows) which uses "GetThreadContext" which is in kernel32.dll.
I noticed that I could get it's address with
unsigned long address = (unsigned long)(&GetThreadContext);
but when I looked at the loaded modules tab - I saw that the symbols for kernel32.dll were not loaded!
How did the VS2008 know the address of "GetThreadContext"?
And how can I do it myself without having the PDBs?
thanks :)
This works for the same reason that
GetThreadContext(hThread, lpContext);
works. Named functions used in your code must be resolved at link-time, or the link would fail. Whether you are taking their address using & or calling them does not matter. At runtime, the DLL is loaded and the function name then resolves to a specific address in the process.
PDB files are used only to provide enhanced symbolic information during debugging. Normally, they are not used at runtime.
[I can't help thinking I'm missing something about this question. Tell me if this is not your problem.]