Get a Win32 program to request a debugger on startup? - c++

We have a C++ Win32 application which spawns, using Qt's QProcess (undoubtedly a wrapper for CreateProcess()), a secondary 'slave' program.
Unfortunately, when debugging the system with Visual Studio 2008, the debugger does not automatically attach to the spawned process.
I know it's possible to programmatically trigger a debugger breakpoint with __debugbreak(), but is it possible for the slave program to throw up the 'Choose a Program to Debug This' window immediately on startup since at the moment we have to race to manually attach to the new process?

Use Image File Execution Options. You can specify the Visual Studio just-in-time debugger as the default debugger to attach to the process.
If you're into using the command-line debuggers, you can use ntsd -o to automatically debug child processes as well.

Another neat trick that I learnt from reading "Programming Applications for Windows" by Jeffrey Richter
Create a DLL with a call to DebugBreak() in DLLMain() inside DLL_PROCESS_ATTACH case
Add path to your new DLL in the registry (it is a comma separated list):
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Windows\AppInit_DLLs
The image loader will create your process and load all DLLs entered in the registry. This will call DLLmain and your breakpoint will get hit.
Note: To avoid debugging all applications, check the process name with "GetModuleBasename" and call the break point only for processes you are interested.

Related

How can I print debug a C++ Win32 DLL?

I am currently working on a project that relies on injecting a DLL into another process, so (AFAICT) standard debugging tools don't work. What I've been using instead is #define DEBUG(_msg) MessageBoxA(nullptr, _msg, "Debug", MB_OK).
This has the issue that it blocks the current thread until I click on the message box, which screws up my code in certain places and/or freezes the process I injected the DLL into.
So, how should I print a debug message with the Win32 API that I can view elsewhere, and continue running code instantly?
In Visual Studio a running process can be debugged.
In the menu Debug look for Attach to process.
Then breakpoints can be triggered like normal.

When are memory dump files exactly created?

I have configured my windows 7 to create mini dump files on crashes but when my application crashed, no dump file was created. The search for answer left me rather confused as to when are dump files created, when windows crashes or my application crashes?
In my case, I am looking for dump file when my application crashes. I receive a typical crash dialog that states:
TheApp Application has stopped working
Windows can check online for a solution to the problem
-> Check online for a solution and close the program
-> Close the program
-> Debug the program
So can I generate dump file for my application when it crashes? I can't produce this bug on development machine so I want to walk back from dump file. Is there any other option to trace the source of bug (to source code)?
First of all, there are different places to configure a "create a minidump on crash" setting, which are totally different.
You can configure Windows to create a kernel dump file when Windows crashes, i.e. when a Bluescreen of death (BSOD) occurs. This is done in the following screen on Windows 7:
You can configure Windows to create a user mode dump file when an application crashes, i.e. instead of the "Windows Error Reporting" dialog which would normally appear. To do so, and you know that in advance, then configure a Registry key called LocalDumps (MSDN). By default, dumps will be created below %LOCALAPPDATA%\CrashDumps and they will have the naming scheme app.exe.<PID>.dmp.
For the sake of completeness, there might be other triggers. The only sure way to tell is: when the method MiniDumpWriteDump (MSDN) is called.
I'm quite sure that you want option 2 of the above. If you have trouble with it, see whether all the conditions for LocalDump are fulfilled.
The answer given by #antlersoft does not work, for the reasons I have posted in my blog: at the time the dialog is shown, Windows has triggered a breakpoint to stop the application and it has injected a callstack of Windows Error Reporting. All in all, not a good starting point for debugging.
What would work is:
attach a debugger of your choice
press "Go" in the debugger
press the "Debug" button of the WER dialog
confirm the warning about the debugger which is already attached
click "No" when asked to start debugging using the selected debugger
Using Task Manager to create a crash dump is not recommended, since it will not consider the bitness of the application, which may cause trouble later. See ways to create good and useful crash dumps.
Minidump is created when Windows crashes. It's not intended to application crash.
If you want to debug crashes of your application, you may attach it to a debugger after it is started. Clicking on the "Debug" button when application crashes do the same. You can use the debugger of MS Visual Studio to do that, for example.
See this page for help on attaching a process to MS Visual Studio debugger:
https://msdn.microsoft.com/en-us/library/3s68z0b3.aspx
EDIT: following text removed, as this may not work as expected (comment from Thomas)
You can also create a dump file from task manager, however you will still need a debugger to analyze it and, actually I am not sure you will be able to get the dump file at the point application crashes. The best way, if you can, is to debug the process on the target machine by attaching it to debugger either after it is started or when crash occurs.
When you get the crash dialog, go to Task Manager, find the process, right click on the process, and select "Create dump file". The dump file is created in the AppData/Local/Temp folder for the user; it will be named %AppData%\Local\Temp\.DMP; if you create multiple it will be -1.DMP, etc. You can move the dump file to your development machine and open it within Visual Studio. Visual Studio will then act as if you had hit "Break all" at the point of the crash while running the process in the debugger.

AQTime DLL profiling - no results

I am trying to profile memory allocation in a DLL, say pluginDLL.dll. I create a new project in AQTime and add the module 'pluginDLL.dll'. This DLL is compiled in Release x64 with VisualStudio 10, all my projects are C++. This DLL is linked to a lib. Say algo.lib. I add to the AQTime project the DLL algo.dll.
pluginDLL.dll is a DLL programmed on an API to an application, say MainApp.exe. If I launch MainApp.exe, then open my plugin, the process MainAppSDK starts. This process starts when any plugin written with this SDK starts.
So, to run the profiling, I proceed as follows (I have already created the project with the two DLLs): I launch manually MainApp.exe. I open manually my plugin, so that MainAppSDK process starts. In AQTime, I attach the process MainAppSDK, and runs the profiling.
If I check Check Memory Bounds, application crashes. Warnings in Event pane are of type 'Access violation' in some msvcr100.dll.
If i check two other options only, i can run my plugin to the end. I can read messages from my application in the event pane, stating that application ran to the end successfully. Before it ends, I click the camera to get the results. And then results are generated but are empty...
Could you help me troubleshoot?
Try starting you profiling using the approach described in the Profiling Multiple Processes help topic instead of just attaching to the running process.
If this does not help, try using the online AQtime Troubleshooter.

Can the visual studio debugger still be used to debug ppapi trusted plugins when using the --ppapi-out-of-process flag?

I am using the new MessageLoop class introduced in pepper-25 so I can run a background thread with blocking ppapi calls for File IO. Up until now I have been running chrome from visual studio with the flags --single-process and --register-pepper-plugins so I can debug my plugin from within visual studio.
Using these flags I found that the call to PPB_GetInterface get_browser for PPB_MESSAGING_INTERFACE was returning null and after some searching I found this issue which states you must run with the flag --ppapi-out-of-process to get MessageLoop support.
With that flag added get_browser does return a valid interface pointer but I can no longer debug my plugin with the visual studio ide as it cannot attach to the child process that my plugin is run from. Is there anyway to tell it to attach to my plug-in process or a way of running from a single process with support for MessageLoop?
Thanks,
James
To get Visual Studio to attach to child processes automatically, you can use the workarounds described at Can Visual Studio be made to debug child processes like WinDBG?.
A different approach is to use the flags --no-sandbox and --ppapi-startup-dialog when you start Chrome. This will cause a message box to appear with the process id when a plugin process is created. You can then connect the Visual Studio debugger to that process using Debug -> Attach to process... -> pick process id. Of course, you could have attached to the process without the Chrome flags and it's not automatic, so this really just gives you certainty that you're attaching to the right process if you have many running at the same time.
A third approach is to use the free Microsoft-provided debugger WinDbg. It is complex and much less user-friendly than the built-in debugger in Visual Studio, but it does have the ability to attach to child processes automatically. Download is available from http://msdn.microsoft.com/en-us/windows/hardware/gg463009.aspx.
As for MessageLoop it will not be supported for in-process plugins. As the issue you link to mentions, developers should move to out-of-process, even if it adds the inconvenience of not being able to attach the Visual Studio debugger automatically without workarounds.

Debugging shell extension in Windows 7

I'm trying to debug shell extension (IContextMenu) in Windows 7 with Visual C++ 2008. I have set DesktopProcess=1 in the registry and set host app to explorer.exe. But when I start the debugger, it launches explorer.exe and then detaches from the process. DllMain of the shell extension isn't called.
The same code with exactly the same settings launched in debugger without any problems in Windows XP + Visual C++ 2008.
Any thoughts how to debug the shell extension in Win7?
I've found a nice workflow that I think is the fastest way to rapidly iterate the code-build-test cycle when developing shell extensions. The following should work on any Windows version.
First prepare - set the start program of your shell extension project to be c:\windows\explorer.exe and also set it to be the start-up project.
Then, whenever you want to debug your shell extension perform the following steps:
Click on the task bar and press Alt-F4 - this will bring up the shut down dialog
Press Ctrl-Alt-Shift-Escape - this combination will close explorer.
Use Alt-Tab to go back to Visual Studio and press F5 - explorer will now launch with the VS debugger attached to it from the very beginning.
When done, just stop the debugger session. This will kill the debugged instance of explorer and will also automatically start a normal instance of it. This will also unlock the shell extension DLL so that you can build it again.
Caveat on Vista and 7 - be sure to run the Visual Studio that you use for debugging in non-Administrator mode (non-elevated), so that the explorer is started in its usual non-elevated mode.
Try launching explorer and THEN attaching the debugger to it.
You could try putting a DebugBreak() call in your code. This should launch the just-in-time debugger at the call and give you an idea of what is going on.
You should take a look at gflags.exe, part of the standard debugging tools sdk. It's got all the options you need to configure (global)flags for any process startup/services/heap/pool-tagging/stacktrace's-on-allocation etc...
Debugging Explorer.exe is usually overkill for extensions that operate in a shell view.
I personally use a little app I made that hosts an instance of IExplorerBrowser similar to this example. If your IContextMenu item is not the default item then you can just use Notepad.exe and its open file dialog...