Headless debugging on Windows - c++

There is a bug that I would like to fix that only occurs on Windows Server without a GUI running. I have set up a Windows Server 2019 machine on Google Compute Engine that reproduces the bug, and would like to debug it.
Ideally, I would like to use gdb, but seeing as the program was built with Visual Studio 2019, gdb can't read the debugging symbols.
I don't have a Windows machine, so using Visual Studio will be difficult. I could set up a VM, but if there's an in-terminal way to do this that would be preferred.
I did a pretty thorough Google search, but it didn't turn up anything. Is there really no Windows solution for debugging C++ code headlessly?

MS has 2 console debuggers called CDB and NTSD so you don't actually need Visual Studio GUI to do the debugging. In fact there are a lot of debugging environments in Windows from MS beside the usual Visual Studio. Just install them in your Windows Server and control them remotely from your terminal
You can also debug MSVC-compiled code with LLDB since the PDB format has been published long time ago and LLVM on Windows does support it. No idea about current LLDB on Linux though
And since you have the source code, sometimes the old-school printf debugger is the best way to analyze the issue
If you can get a Windows VM it'll be much better to do remote debugging. In fact almost all debuggers support that feature including GDB or LLDB, so even if you don't have the source code you can still run any Windows debugger and step through the instructions instead of high-level code lines from a remote machine
An alternative way is to take a memory dump and debug later. After getting the dump file, just drag it into your VS solution or any debugging tool like WinDbg and then select "Start Debugging". Now you can step through instructions/code lines and examine variables' values, or jump to an arbitrary function's stack frame just as if you're really running the malfunctioning app
There are many ways to dump a process' memory. You can set Windows to automatically save a dump file when your app crashes, or just capture memory snapshots manually during runtime. Comparing 2 snapshots is also useful for detecting leaked memory. For more information on how to do that read
Collecting User-Mode Dumps
Steps to Catch a Simple “Crash Dump” of a Crashing Process
There's also an easy way to take a dump of a live process using task manager (or any other similar tools)

Related

Embedding thread names in Windows minidump files

Our 32-bit C++ application names its threads using the standard approach on Windows. We generate minidumps when errors occur, but when the dumps are loaded into Visual Studio 2013 the thread names are missing. This makes it harder to debug the problem given the high number of threads present.
The only way I've heard of to improve this comes from this Microsoft User Voice posting, where the caller suggests embedding them in a custom stream, and writing a custom Visual Studio extension to rename the threads upon loading the dump. However, this seems cumbersome and apparently adds a lot of time to the loading process.
Is there an easier way to embed thread names in the dump so they "just work" when loading them in Visual Studio?
For reference: we use Visual Studio 2013 to build, our apps run on Win7 or higher, and we use Google Breakpad to generate dumps.
Microsoft has indicated that the SetThreadDescription API will be the basis for any proper support they add in the future for thread names in dumps, debuggers, and/or analysis tools.
Unfortunately it appears that this API is rather new and is not yet supported in dumps, WinDbg, Visual Studio Debugger, etc. There is a User Voice request to add this support, so please vote for it if you would like a way to properly give threads a persistent name.
After some more digging, it seems like one way to do this would be to store a pointer to the name in the ArbitraryUserPointer field of the TEB of the thread.
If the TEB was then embedded in the minidump (which can be achieved with the MiniDumpWithProcessThreadData flag, according to this article) then it seems like at the very least we could view the current thread name in the Visual Studio watch window by evaluating
(*(char **)(#TIB+0x14))
Perhaps a Visual Studio debugger add-on could be written to examine the contents of the TEB when a process is attached to, and use the exception 'hack' to set the name of the thread in the Threads window.

Get stack trace of a crash on Windows without installing Visual Studio? (C++)

I have an C++ application, that crashes on a computer of some person on the other end of the world. There's no way for me to simulate it or get the same computer. The person is no developer, so I cannot really ask him to install Visual Studio or something. I have pretty deep debug logs, but they didn't reveal anything usable.
Is there a tool, that could generate the stack trace of the application at the moment of the crash? Such thing is available inside OSX, but seems that Windows doesn't have it.
You can use procdump. It can be setup as a debugger to automatically create dumps for crashing processes.
Procdump is part of Sysinternal tools and can be found at:
http://technet.microsoft.com/en-us/sysinternals/dd996900.aspx
Relevant switches:
Create a dump for a hung application:
Write a mini dump for a process named 'hang.exe' when one of it's
Windows is unresponsive for more than 5 seconds:
C:\>procdump -h hang.exe hungwindow.dmp
Automatically create dumps for crashing apps:
Register as the Just-in-Time (AeDebug) debugger. Makes full dumps in
c:\dumps.
C:\>procdump -ma -i c:\dumps
Create a dump for a pid:
C:\>procdump <PID>
You can read the dump files using windbg: Getting started with dump file analysis
Use google's lib that makes minidump for msvc to debug.
CrashRpt
Since google seems to lead here for all "stack traces of windows" you can also get the stack trace of a running process using "sleepy" or "very sleepy" utilities.
another option: run it in gdb.exe, hit ctrl+c

My C++ executable runs way faster outside the Visual Studio 2008 IDE than inside, even in release

I build a C++ application that does some number crunching.
I'm running in Visual Studio 2008 PRO SP1, in release mode, Windows 7 64 bit.
If I run it inside the IDE, the application takes 4 minutes, if I run the same executable from windows explorer it takes 6 seconds!
I have no clue. I have checked that this does not depend on the processor and operating system. I don't think I have strange VS plugins that are doing something in the background.
Any hints?
Thank you in advance!
Marco
Presumably, the slow down is caused by the debugger being attached when you are starting the application in Visual Studio. This is the case even when you've built the program in "Release" mode.
To confirm that this is indeed the source of your problem, try running your application without the debugger, using the "Start Without Debugging" command or Ctrl+F5.
It's worth nothing that in C++ specifically, when you start without debugging, your program won't use the Windows debug heap. With the debugger attached, it will.
As Cody mentioned, one option is to simply not debug. But if you want to speed up your debugging sessions, here are a few things I've found can make a huge difference:
Remove debugging print statements that are no longer necessary. If you see your log filling up with text, that is likely slowing you down significantly.
Remove breakpoints (Ctrl+Shift+F5). A couple times I've noticed a huge drop in performance, and it turned out to be due to a breakpoint with a condition that was never met.

How can I debug an application crash in Win7 after it's happened?

I have a Visual Basic 6 application that I've recently changed to use a couple of C++ DLLs I've written in Visual Studio 2008. The application works fine on my PC, but when we install it on one of our test PCs it tends to crash during shutdown - we see the Win 7 message "Your application has failed" or whatever it is.
I know Win 7 stores data that can be used to analyse the crash. I've got the source code and .PDB files from the build so I should be able to use that, but I can't figure out where Win 7 stores the data from the crash. The Event Viewer shows the crash but doesn't have any data and the directory C:\Windows\Minidump doesn't exist.
Where do the crash files get put?
Microsoft has documentation here which works back to Vista Service Pack 1:
https://msdn.microsoft.com/en-us/library/windows/desktop/bb787181(v=vs.85).aspx
There is a registry key with 4 sub-values that control generic crashes:
# HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps
DumpType # 1 = mini-dump, 2 = full dump, 0 = custom dump
DumpCount # Maximum number of dump files to keep before purging
DumpFolder # Full path to folder to store dump files (no trailing slash)
DumpFlags # Usually 0, flags only used if DumpType is 0
Once these registry values are in place crash files should be immediately written no matter what happens to whatever dialog might be displayed.
Note: Registry sub-keys with executable name can be used to control crash behavior for only one specified process.
Get minidump for crash
Get WDK , Using Debugging Tools for Windows
Use Windbg to open crash dump
use command !analyze
The !analyze extension displays information about the current exception or bug check.
Read more about window debugging
Creating Dump:
You can configure windbg as defaut debugging tool by giving command "windbg -I" in run.
Also Dr. Watson tool can do this for you.
You can even use a carsh reporting mechanism and get the locally saved dump file, then debug it using Visual Studio. There are pretty much freely available resources in Visual C++, but not much in VB.
Why don't you make your program save minidump wherever you want when the crash happens? I'm not familiar with VB, but try to use SetUnhandledExceptionFilter() and MiniDumpWriteDump().

Visual studio release build

I'm trying to generate a release build for a C++ application that I've written. The application runs fine (debug & release) when you run it from within VS2008; but when you run the executable it crashes nearly every single time.
Now, is there a hack so I can run this application as a standalone application without having to run through all of the code and finding the bug that is causing it?
Thanks in advance.
In short, no.
you will have to find the bug, if it works within VS, then I'd hazard a guess that it is a timing issue, possibly you're overwriting shared thread data, this would be less likely (though still possible to see) inside VS as its being run in a debug environment which slows it down a bit.
If you want help finding your bug, then tell us more. Otherwise, build your release with debug symbols (pdbs), install DrWatson as the system debugger and run it standalone. When it crashes DrWatson will create a minidump file, load this into WinDbg (my favourite) and you'll be able to see exactly where your bug is (it'll even tell you that the dump contains an exception and show you it by default. You need to add your source code path and path to your symbols in WinDbg to get it to do this correctly).
Then you will also know how to diagnose crashes when the app is run on-site too.
Are you loading external resources? If you are check that your relative paths are correct in the C++ program.
One possibility is that your program uses uninitialized heap data. Launching a program from the debugger enables the NT debug heap, which causes the heap allocator to fill new memory blocks with a fill pattern, and also enables some heap checking. Launching the same program from outside the debugger leaves the NT debug heap disabled, but if the program was linked against the debug version of the C runtime, then the CRT debug heap will still be enabled.
A much less likely possibility is that your program requires SeDebugPrivilege to be set in its process token. The debugger enables this privilege in its process token, which has the side effect that all programs launched from the debugger inherit this privilege. If your program tries to use OpenProcess()/ReadProcessMemory()/WriteProcessMemory() and doesn't handle errors correctly, it's conceivable that it could crash.
There are a few possibilities. Besides what has already been mentioned, running an app from Visual Studio will execute in the same security context as the Visual Studio instance. So if, for instance, you are working on Vista, you might be hitting an unhandled security violation if you're trying to access protected files, or the registry.
What if you build a debug version and run it standalone? Does it crash? If so, you can usually break into the debugger from there and get a call stack to see what the malfunction is.
From the details you've given, it sounds like there may be a library issue. Are you running the program on the same computer? If not then you'll also have to deploy the appropriate libraries for your application. If you are running on the same computer but outside of the dev environment, ensure that your application can see the appropriate libraries.
Best way i have found to debug in release is to create a crash dump when an crash happens and the dump then allows me to load debug symbols on my dev computer and find out whats going on. More info here: http://www.debuginfo.com/articles/effminidumps.html
You can also go to file => open in Visual Studio and open the .exe, so you are not starting it under the debugger per se. Not sure if it will help.
http://blogs.msdn.com/saraford/archive/2008/08/21/did-you-know-you-can-debug-an-executable-that-isn-t-a-part-of-a-visual-studio-project-without-using-tools-attach-to-process-296.aspx