How can I print debug a C++ Win32 DLL? - c++

I am currently working on a project that relies on injecting a DLL into another process, so (AFAICT) standard debugging tools don't work. What I've been using instead is #define DEBUG(_msg) MessageBoxA(nullptr, _msg, "Debug", MB_OK).
This has the issue that it blocks the current thread until I click on the message box, which screws up my code in certain places and/or freezes the process I injected the DLL into.
So, how should I print a debug message with the Win32 API that I can view elsewhere, and continue running code instantly?

In Visual Studio a running process can be debugged.
In the menu Debug look for Attach to process.
Then breakpoints can be triggered like normal.

Related

OutputDebugString function don't send the message in my console app

I try to send debug mesage with OutputDebugString function. I use this function in my dll project and i get this message in DebugView. I also use this function in my console app project(same solution) but i can't get messages. All messages appear in the DebugView except for the console application. It does not even appear the simple OutputDebugStringA("debug"); or OutputDebugString(L"debug"); codes. What could it cause?
One issue is that whether you visit it in your VS output window or the tool DebugView here: https://technet.microsoft.com/en-us/sysinternals/bb896647?f=255&MSPPError=-2147217396. They are different.
If you use the Debug view, not debug your app, just run it using "start without debugging(Ctrl+F5)".
If you use the VS output window and other apps output normally, just the specific console app has this issue, I don't think it is related to the debugger tool, we would think about the project itself or VS Settings.
(1)Please set a breakpoint in the code line "OutputDebugString(L"debug");", and then debug your app using "F5", after the breakpoint was hit, click "Step Into (F11)", and then visit your debug output window. Maybe the specific line code was not called in your side.
(2)Since there are different projects in the same solution, please set this console app as the start up project. Or you could just create a single solution for this project, debug it again. At least, we could know that whether other projects impact this specific console app.
(3) Just test it in other VS IDE, make sure that it is not the VS IDE issue. I just use the default VS settings in my side, it works well.
Reference:
https://blogs.msdn.microsoft.com/reiley/2011/07/29/a-debugging-approach-to-outputdebugstring/

When are memory dump files exactly created?

I have configured my windows 7 to create mini dump files on crashes but when my application crashed, no dump file was created. The search for answer left me rather confused as to when are dump files created, when windows crashes or my application crashes?
In my case, I am looking for dump file when my application crashes. I receive a typical crash dialog that states:
TheApp Application has stopped working
Windows can check online for a solution to the problem
-> Check online for a solution and close the program
-> Close the program
-> Debug the program
So can I generate dump file for my application when it crashes? I can't produce this bug on development machine so I want to walk back from dump file. Is there any other option to trace the source of bug (to source code)?
First of all, there are different places to configure a "create a minidump on crash" setting, which are totally different.
You can configure Windows to create a kernel dump file when Windows crashes, i.e. when a Bluescreen of death (BSOD) occurs. This is done in the following screen on Windows 7:
You can configure Windows to create a user mode dump file when an application crashes, i.e. instead of the "Windows Error Reporting" dialog which would normally appear. To do so, and you know that in advance, then configure a Registry key called LocalDumps (MSDN). By default, dumps will be created below %LOCALAPPDATA%\CrashDumps and they will have the naming scheme app.exe.<PID>.dmp.
For the sake of completeness, there might be other triggers. The only sure way to tell is: when the method MiniDumpWriteDump (MSDN) is called.
I'm quite sure that you want option 2 of the above. If you have trouble with it, see whether all the conditions for LocalDump are fulfilled.
The answer given by #antlersoft does not work, for the reasons I have posted in my blog: at the time the dialog is shown, Windows has triggered a breakpoint to stop the application and it has injected a callstack of Windows Error Reporting. All in all, not a good starting point for debugging.
What would work is:
attach a debugger of your choice
press "Go" in the debugger
press the "Debug" button of the WER dialog
confirm the warning about the debugger which is already attached
click "No" when asked to start debugging using the selected debugger
Using Task Manager to create a crash dump is not recommended, since it will not consider the bitness of the application, which may cause trouble later. See ways to create good and useful crash dumps.
Minidump is created when Windows crashes. It's not intended to application crash.
If you want to debug crashes of your application, you may attach it to a debugger after it is started. Clicking on the "Debug" button when application crashes do the same. You can use the debugger of MS Visual Studio to do that, for example.
See this page for help on attaching a process to MS Visual Studio debugger:
https://msdn.microsoft.com/en-us/library/3s68z0b3.aspx
EDIT: following text removed, as this may not work as expected (comment from Thomas)
You can also create a dump file from task manager, however you will still need a debugger to analyze it and, actually I am not sure you will be able to get the dump file at the point application crashes. The best way, if you can, is to debug the process on the target machine by attaching it to debugger either after it is started or when crash occurs.
When you get the crash dialog, go to Task Manager, find the process, right click on the process, and select "Create dump file". The dump file is created in the AppData/Local/Temp folder for the user; it will be named %AppData%\Local\Temp\.DMP; if you create multiple it will be -1.DMP, etc. You can move the dump file to your development machine and open it within Visual Studio. Visual Studio will then act as if you had hit "Break all" at the point of the crash while running the process in the debugger.

AQTime DLL profiling - no results

I am trying to profile memory allocation in a DLL, say pluginDLL.dll. I create a new project in AQTime and add the module 'pluginDLL.dll'. This DLL is compiled in Release x64 with VisualStudio 10, all my projects are C++. This DLL is linked to a lib. Say algo.lib. I add to the AQTime project the DLL algo.dll.
pluginDLL.dll is a DLL programmed on an API to an application, say MainApp.exe. If I launch MainApp.exe, then open my plugin, the process MainAppSDK starts. This process starts when any plugin written with this SDK starts.
So, to run the profiling, I proceed as follows (I have already created the project with the two DLLs): I launch manually MainApp.exe. I open manually my plugin, so that MainAppSDK process starts. In AQTime, I attach the process MainAppSDK, and runs the profiling.
If I check Check Memory Bounds, application crashes. Warnings in Event pane are of type 'Access violation' in some msvcr100.dll.
If i check two other options only, i can run my plugin to the end. I can read messages from my application in the event pane, stating that application ran to the end successfully. Before it ends, I click the camera to get the results. And then results are generated but are empty...
Could you help me troubleshoot?
Try starting you profiling using the approach described in the Profiling Multiple Processes help topic instead of just attaching to the running process.
If this does not help, try using the online AQtime Troubleshooter.

how to attach to a process that loads my DLL and debug it

basically, I am developing a DLL (an open interface) that will be used by a third party software. I want to debug my DLL after it is loaded by this thirdparty DLL, because I found a specific bug that only happens when my DLL is loaded by this third party software (my DLL works OK for all other software that accepts this open interface)
I set __debugbreak() in my the code DLL and run the third party software. An Visual studio dialog did jump up asking if I want to debug.. but after I click OK, instead of getting get into a visual studio window that shows the line of code that I set a breakpoint, visual studio just crashes.
I found out that the third party softward loads my DLL and runs it in another thread. And I suspect this is the reason that the third party software crashes, because effetively __debugbreak() throws an uncaught execption in a thread launched by the main thread.
In this case, how can i debug my DLL?
It's unfortunate that VS crashes, but you might find a way around that.
Best to start the debugging session right ahead. Or use 'attach to process' when the process is blocked on a messagebox or some other input. Also better use breakpoints.
With __debugbreak() or forced access violation try to ask a new instance of VS not an existing one.

Get a Win32 program to request a debugger on startup?

We have a C++ Win32 application which spawns, using Qt's QProcess (undoubtedly a wrapper for CreateProcess()), a secondary 'slave' program.
Unfortunately, when debugging the system with Visual Studio 2008, the debugger does not automatically attach to the spawned process.
I know it's possible to programmatically trigger a debugger breakpoint with __debugbreak(), but is it possible for the slave program to throw up the 'Choose a Program to Debug This' window immediately on startup since at the moment we have to race to manually attach to the new process?
Use Image File Execution Options. You can specify the Visual Studio just-in-time debugger as the default debugger to attach to the process.
If you're into using the command-line debuggers, you can use ntsd -o to automatically debug child processes as well.
Another neat trick that I learnt from reading "Programming Applications for Windows" by Jeffrey Richter
Create a DLL with a call to DebugBreak() in DLLMain() inside DLL_PROCESS_ATTACH case
Add path to your new DLL in the registry (it is a comma separated list):
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Windows\AppInit_DLLs
The image loader will create your process and load all DLLs entered in the registry. This will call DLLmain and your breakpoint will get hit.
Note: To avoid debugging all applications, check the process name with "GetModuleBasename" and call the break point only for processes you are interested.