How to debug an external process under C++ Builder? - c++

I have a DLL that is written in C++ Builder. The DLL is built with "debug info" on.
I cannot run and debug this DLL from the IDE because it is loaded dynamically as a plugin in another process (main exe). My DLL needs to be copied into the main exe's folder first.
So, I attached the IDE to that process but I don't know what to do from here.
I know that if I click a button in the main process to load and use my DLL an AV is raised and a custom message is shown with some minor into about the error. The debugger won't step in when the AV is raised. Probably the error is caught at a higher level.
It would be nice if I could also attach the source-code of my DLL and put a breakpoint in it.
How do I get more info about that AV (its origins)?
(Basically any hint from those that did this type of debugging would be very helpful).

I have a DLL that is written in C++ Builder. The DLL is built with "debug info" on.
I cannot run and debug this DLL from the IDE because it is loaded as a plugin in another process.
Yes, you can.
Open the DLL project in the IDE, go into the project's Run parameters and set the desired EXE as the project's Host. This way, when you "run" the DLL project for debugging, the IDE will execute the Host instead and attach the debugger to that process. When the Host process loads your DLL into memory, you can then step through and debug the DLL's code as needed.
If the Host process is already running before you start your debugging, you can simply Attach the debugger to the Host process manually before it loads your DLL, and then the debugger will still be able to step through the DLL's code once the Host loads the DLL into memory.

Related

How to debug a DLL project using QtCreator?

I have written a non-Qt C DLL compiled with mingw using the QtCreator IDE.
When I inject the DLL into a process, the DLL causes that process to crash at a certain line of code. I found this line to be the culprit through the use of OutputDebugString. I know how to fix the line and the DLL works when that line is modified. However, for the purpose of learning how to use a debugger, I have left the line broken and unmodified.
How would I use the QtCreator debugger to find that same exact line is causing the problem? When QtCreator is set to compile in debug mode, pressing F5 results in the following dialog because there is no EXE:
The solution is to run the debugger attached to the executable before injecting the DLL. The DLL must be built in debug configuration and its project needs to be opened. Attach the debugger using instructions here and inject the DLL. If executable crashes you should see the stack trace and your DLL code when you click on selected stack frame. But be careful: if you corrupted the memory of the process the stack trace might be incomplete or incorrect.

MS Visual C++ Runtime Library error on launch - any debugging tricks?

When launching my app, I'm getting the below error dialog. I understand this indicates a problem loading the runtime library. The problem is, I'm not seeing any way to get more specific info. Which library? What was the exact problem it had when loading? etc.. System event viewer doesn't have any entries for it. Are there any tricks to finding out exactly which library it was trying to load when it hit the error and what the specific problem was?
Microsoft Visual C++ Runtime Library
Runtime Error!
Program: exe path
R6034
An application has made an attempt to load the C runtime library incorrectly.
Please contact the application's support team for more information.*
My current approach for dealing with runtime lib dependencies is to scan the install directory, extracting out the embedded manifests for the .dll and .exe files and then generating .config and .2.config files to re-direct to the minor versions of the runtime lib I'm shipping with. In the past this has always gotten things working. Not so in this case.
This might be complicated somewhat by the fact that the calling code is native C++ but some of the DLLs use .NET. I'm calling a C++ API, but under the hood it ends up utilizing some DLLs which themselves use .NET.
Use SysInternals Process Monitor to monitor file system access. Filter on process name and operation (CreateFile) to see what DLLs the process can't locate.
It could indicate a wrong version of msvcr*.dll.
You could try Dependency Walker (http://www.dependencywalker.com/) to find out where it's being linked.
Make sure you have a debug version of the code you're trying to attach to. On the Debug menu, select Attach to Process. Use processID -
Also check this -http://support.microsoft.com/kb/235434
Resolution 1
You should be able to attach to the process using Task Manager:
Enable just-in-time debugging (JIT):
In Visual C++, on the Tools menu, click Options.
On the Debug tab, make sure that Just-in-time debugging is selected.
Run Task Manager and select the process to attach to. Right-click and select Debug.
Resolution 2
Run Task Manager and get the process ID for the process you want to debug.
At a command prompt enter the following:msdev -p
This will start Visual C++ and will attach to the process specified.

debugging exe and dll project in carbide 2.7

I'm running carbide 2.7 with 9.3 SDK FP2. I have two projects, one is an animation dll TARGETTYPE ANI and another is an exe that will invoke it.
what is the way that i should debug this in the emulator? what do i need to change so both the exe and dll are deployed in the emulator and i get to debug them?
Well I don't have access right now to a working Carbide, but you will need surely the followings:
load both projects to your workspace
at the debugger configuration (something like Debug... or Debug settings, I don't remember), there is a "load symbols" options and then you have to select "from all projects in the workspace" or at least tick your two projects.
sometimes this does not work. You can try to load the symbols at runtime, there will be a "load symbols from" somewhere in the menu system, you have to select your dll binary here.
try to put a breakpoint at the entry point of your dll
if you debug on the device ensure that you copy the debug variant of your binaries into the sis
Debugging multiple binaries in carbide is a bit of matter of luck, sometime you succeed sometime you don't as this part of carbide has some bugs. However it is definitely possible, I did it already in the past :)

Debugging InProc COM Dll

I have a project in VC++ 6.0 where there is an exe and a InProc COM Dll. I want to be able to place a breakpoint somewhere in the InProc COM DLL, but VC++ won't allow me to set a breakpoint.
I have the source code for this DLL, however I cannot figure out how I can place a breakpoint in the code and the debug it?
Can someone help me.
It has been some time since I worked with COM but IIRIC, inside your COM project configure your executable as the launching application. It should work (sorry, I don't have VC++ 6.0 installed here anymore :().
If it doesn't work, you can try to attach the debugger to the running application.
In either cases make sure you have full debug information in your COM server.
Hope this helps.
Attach to the process
Open Project->Settings (Alt+F7)
Open Debug tab, category Additional DLLs
Add you in-proc server DLL
Save .opt file on closing the debugger
This way next time you attach to process or manually open the .opt file, your in-proc server DLL gets loaded, its PDB gets parsed, last open source files get loaded, breakpoints get loaded.
The reason why "additional dlls" setting is needed here is because you in-proc server doesn't get loaded until an instance of his is CoCreated. So the debugger doesn't load its PDB file and the source files are treated as unknown text files, so the breakpoints in them get inactive (white).
Two things you can look into
Uncheck Require source files to exactly match the original version in the Debugging options dialog
If that fails, compile the DLL again (preferably with optimizations disabled /Od) and use the new DLL with its PDB file.
Not sure if this will work in VC6 but you could try _asm int 3 where you want the code to break, this should cause a breakpoint in your code and allow you to debug it.

Why is my DLL failing to register?

I am building a project in VS2005 and several of my DLLs are failing to register. The error message I am getting in Visual Studio is:
Project : error PRJ0019: A tool returned an error code from "Registering ActiveX Control..."
which is nicely vague. When I register the DLL manually through the command line (using regsv32.exe, I get the following error:
LoadLibrary("test.ocx") failed - This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix the problem.
I ran Dependency Walker (depends.exe) on the culprit .ocx file but it isn't showing any obvious problems.
I have also done a re-build but I'm still getting the same problem.
Any suggestions as to how I could determine the cause of this failure to register?
Microsoft had recently released a Security Update for ATL (KB971090). It is un update on top of MSVS2005sp1 and it's both compilate-time and runtime compatibility breaker. Check if your building environment has this patch.
References:
ATL Security Update:
http://msdn.microsoft.com/en-us/visualc/ee309358.aspx
Breaking changes in ATL:
http://msdn.microsoft.com/de-de/library/ms235654.aspx
And this is a must read:
http://tedwvc.wordpress.com/2009/08/10/avoiding-problems-with-vc2005-sp1-security-update-kb971090/
Most probable is because the embedded manifests. You should take a resource explorer application and check your DLLs for the embedded manifests. It might be that one of the dependent DLLs (or your DLL) require some versions of other DLLs which don't exists.
I got this message: "This application has failed to start because the application configuration is incorrect." in case of embedded manifest mistmatches.
Probably the easiest way to troubleshoot this whole category of problem is to install Process Monitor from microsoft.com.
Process Montior lets you observe the system calls processes are making, and in this case you will be able to see if some system call is failing. For example, if you are lacking a dependency, then a CreateFile() call will be seen failing with a DLL, OCX, etc. as the filename.
Launch procmon and configure the filter to exclude events other than from regsvr32.exe, reproduce your test case, and then check the log. Look for NAME_NOT_FOUND errors in the return value column.
Do you have the C++ Redistributable Package Installed?
There are several things you can try:
try regsvr32 w/ fusion log enabled (fuslogvw.exe - it works for unmanaged dlls as well). This would give you a bit more information than depends on what external dependencies are loaded, where are they loaded from and what errors were hit.
copy the .ocx and its dependencies to the root or a first level folder and try registering from there. I don't remember details, but there was an old problem with registering a COM dll from within too deep of a path.
run regsvr32 under WinDbg. Set a breakpoint DllMain and see if it does anything funky.
If you never break on DllMain in WinDbg, set a breakpoint on module load for your dll and once it's hit, you can either step through LoadLibrary, or just set a generic load library breakpoint and check every dll after that.