I'm studying kernel mode driver following to this Youtube video and preparing for debugging a driver in a VirtualBox VM, with WinDbg and Virtual KD.
I set up the symbol file by clicking
File / Symbol file path
add symbol path
SRV*c:\symbols* http://msdl.microsoft.com/download/symbols
put a check mark to "reload" item
click ok
After that, Windbg's screen is as follows:
************* Path validation summary **************
Response Time (ms) Location
Deferred SRV*c:\symbols* http://msdl.microsoft.com/download/symbols
kd> .reload
Connected to Windows 10 17134 x64 target at (Sun Oct 7 13:16:30.147 2018 (UTC + 9:00)), ptr64 TRUE
Loading Kernel Symbols
...............................................................
................................................................
..........................
Loading User Symbols
Loading unloaded module list
......Unable to enumerate user-mode unloaded modules, Win32 error 0n30
I can not understand symbol files functioning properly.
Are Symbol files currently not available?
I use lml command in such case.
If symbol files are loaded, you can find module name like this.
2: kd> lml
start end module name
ffff9e54`ba960000 ffff9e54`ba9d7000 win32k (pdb symbols) c:\symbols\win32k.pdb\901A464ABCFD2696F50FFB02C607B4661\win32k.pdb
fffff803`6921a000 fffff803`69aef000 nt (pdb symbols) c:\symbols\ntkrnlmp.pdb\9378084E8DBD4AB1A155099BCE693E341\ntkrnlmp.pdb
Related
On Linux and Mac, when using dlopen() to load a shared library that links to another library, if linking fails because of a missing symbol, you can get the name of the missing symbol with dlerror(). It says something like
dlopen failed: cannot locate symbol "foo"
On Windows, when using LoadLibrary() to load a DLL with a missing symbol, you can only get an error code from GetLastError() which for this type of issue will always be 127. How can I figure out which symbol is missing, or a more verbose error message from LoadLibrary() that explains why the function failed?
I figured out a way using the MSYS2 terminal. Other methods might work with GUI software.
A major caveat is that this can't be done in pure C/C++ and released for end users. It's for developers only, but it's better than nothing.
Install Debugging Tools for Windows by downloading the Windows SDK and unchecking everything except Debugging Tools.
I could be wrong, but it seems that installing this software installs a hook into the Windows kernel to allow LoadLibrary() to write verbose information to stderr.
Open the MSYS2 Mingw64 terminal as an administrator and run
'/c/Program Files (x86)/Windows Kits/10/Debuggers/x64/gflags.exe' -i main.exe +sls
This prints the following to the terminal to confirm that the registry has been changed.
Current Registry Settings for main.exe executable are: 00000002
sls - Show Loader Snaps
Use -sls instead of +sls if you need to undo, since I believe that the change takes place for all programs called main.exe in Windows globally, not just for your file.
Then running main.exe should print debug information to stderr, but since I'm debugging an -mwindows application, it's not working for me.
But for some reason, running the binary with MSYS2's gdb allows this debug information to be printed to stderr.
Install mingw-w64-x86_64-gdb with MSYS2 and run gdb ./main.exe and type run or r.
Search for a section similar to the following.
warning: 1ec8:43a0 # 764081125 - LdrpNameToOrdinal - WARNING: Procedure "foo" could not be located in DLL at base 0x000000006FC40000.
warning: 1ec8:43a0 # 764081125 - LdrpReportError - ERROR: Locating export "foo" for DLL "C:\whatever\plugin.dll" failed with status: 0xc0000139.
warning: 1ec8:43a0 # 764081125 - LdrpGenericExceptionFilter - ERROR: Function LdrpSnapModule raised exception 0xc0000139
Exception record: .exr 00000000050BE5F0
Context record: .cxr 00000000050BE100
warning: 1ec8:43a0 # 764081125 - LdrpProcessWork - ERROR: Unable to load DLL: "C:\whatever\plugin.dll", Parent Module: "(null)", Status: 0xc0000139
warning: 1ec8:43a0 # 764081171 - LdrpLoadDllInternal - RETURN: Status: 0xc0000139
warning: 1ec8:43a0 # 764081171 - LdrLoadDll - RETURN: Status: 0xc0000139
Great! It says Procedure "foo" could not be located in DLL so we have our missing symbol, just like in POSIX/UNIX's dlopen().
While the answer from Remy Lebeau is technically correct, determining the missing symbol from GetLastError() is still possible on a Windows platform. To understand what exactly is missing, understanding the terminology is critical.
Symbol:
When a DLL is compiled, it's functions are referenced by symbols.
These symbols directly relate to the functions name (the symbols are
represented by visible and readable strings), its return type, and
it's parameters. The symbols can actually be read directly through a
text editor although difficult to find in large DLLs.DLL Symbols - C++ Forum
To have a missing symbol implies that a function within cannot be found. If this error occurs prior to using GetProcAddress(), then it's possible that any number of functions cannot be loaded due to missing prerequisites. This means it is possible that a library that you are attempting to load also requires a library that the first cannot load. These levels of dependency may go on for an unknown number of layers, but the only answer that GetLastError() can determine is that there was a missing symbol. One such method is by using Dependency Walker to determine the missing library the first library requires. Once all required libraries are available and can be found by that library (which can be its own can of worms), that library can be loaded via LoadLibrary().
Trying to debug a crash in one of our DLL's. It is loaded into Server Manager and crashes when trying to configure Active Directory Certificate Services (the DLL is a registered provider). I know the crash is an access violation and I have the pdb file, just don't know how to go about debugging this. I've read pages such as this and this (didn't help). I tried to glean the info using windbg (using lm to get the loaded address, which appears to be 8000000:
"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\windbg.exe" -z myKSP.dll
Then
0:000> lm
start end module name
00000001`80000000 00000001`8005e000 ...
Then, since the Event Viewer tells me:
Exception code: 0xc0000005
Fault offset: 0x000000000002a601
I tried to view that:
0:000> ln 80000000+2a601
Browse module
Set bu breakpoint
Nothing is shown.
I have VS2015, so, I tried to attach to the serververmanager.exe process. Next, I tried loading symbols via Tools->Options->Debugging->Symbols and specifying the path, but, when I set a breakpoint, I always receive "no symbols have been loaded". In the previous symbol windows, I set the cache folder, which downloaded a bunch of stuff, but that did not seem to load anything.
Clearly, I'm not using the tools correctly. How do I debug a DLL, compiled in Release mode, PDB is available, that is loaded by the ServerManager.exe or whatever sub-process it might spawn)?
Start windbg, press Ctrl-D to open your dump file, then type the following. That should give you either a significant stack after one of the kp500, or at least will tell you whether the pdb file doesn't match the binary.
.symfix
.sympath+ <FOLDER_WITH_YOUR_PDB>
.reload
!sym noisy
.reload /v /f myKSP.dll
!sym quiet
kp500
.ecxr
kp500
Trying to debug minidump in windbg/VS. Cannot get symbols for msvcr90 of version 9.0.30729.8387. Was trying to do .symfix. If I set sympath for my application symbols, I can get part of call stack which of my code, but not of msvcr. If I use .symfix, partially I can get several calls from c runtime, but, obviously, not from my code.
!sym noisy without .symfix:
SYMSRV: \\eserver\symstore\ms\msvcr90.dll\51EA1BBDa3000\msvcr90.dll not found
SYMSRV: \\eserver\symstore\ms\msvcr90.dll\51EA1BBDa3000\msvcr90.dll not found
SYMSRV: \\eserver\symstore\ms\msvcr90.dll\51EA1BBDa3000\msvcr90.dll not found
SYMSRV: http://msdl.microsoft.com/download/symbols/msvcr90.dll/51EA1BBDa3000/msvcr90.dll not found
DBGHELP: C:\Program Files\Debugging Tools for Windows (x64)\msvcr90.dll - file not found
SYMSRV: \\eserver\symstore\ms\msvcr90.dll\51EA1BBDa3000\msvcr90.dll not found
SYMSRV: \\eserver\symstore\ms\msvcr90.dll\51EA1BBDa3000\msvcr90.dll not found
SYMSRV: \\eserver\symstore\ms\msvcr90.dll\51EA1BBDa3000\msvcr90.dll not found
SYMSRV: http://msdl.microsoft.com/download/symbols/msvcr90.dll/51EA1BBDa3000/msvcr90.dll not found
DBGENG: C:\Windows\WinSxS\amd64_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.8387_none_08e793bfa83a89b5\msvcr90.dll - Couldn't map image from disk.
Unable to load image C:\Windows\WinSxS\amd64_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.8387_none_08e793bfa83a89b5\msvcr90.dll, Win32 error 0n2
DBGENG: msvcr90.dll - Partial symbol image load missing image info
DBGHELP: Module is not fully loaded into memory.
DBGHELP: Searching for symbols using debugger-provided data.
*** WARNING: Unable to verify timestamp for msvcr90.dll
The last thing I can see with this is call from msvcr90!_freefls. In all ways I can get only part of call stack and I need full. Is there a way to obtain right symbols? Similiar problem with symbols for 9.0.30729.6871.
Client OS: Windows Server 2012 (build 9200) 64-bit, Windows 7 (build 7600) 64-bit, Windows Vista (build 6000) 64-bit, Windows Server 2008 R2 (build 7600) 64-bit.
You can have multiple symbol paths so the debugger can get symbols from multiple places.
.Symfix
Is a good first command to run.
Then use
.sympathy+ c:\<My_symbols>
The other problem above is
Couldn't map image from disk.
that comes from
DBGENG: C:\Windows\WinSxS\amd64_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.8387_none_08e793bfa83a89b5\msvcr90.dll - Couldn't map image from disk.
Unable to load image C:\Windows\WinSxS\amd64_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.8387_none_08e793bfa83a89b5\msvcr90.dll, Win32 error 0n2
DBGENG: msvcr90.dll - Partial symbol image load missing image info
DBGHELP: Module is not fully loaded into memory.
I would make another copy of that file and point your symbol path to that to see if the debugger can load it from there.
Solved with reproducing problem on client OS with attached debugger.
I wrote a C++ project called 'Foo' using Microsoft Visual Studio 2005 Verison 8.0.50727.762 (SP.050727-7600) on Windows XP Professional Version 2002 Service Pack 3. I built the project into Foo.exe. Then, I copied the file Foo.exe to a Windows Server 2003 Enterprise Edition Service Pack 2. It runs fine.
Now, I try to debug it using WinDbg:6.12.0002.663 X86 Version 5.2.
No symbol path, source path or image path is set
When I open C:\foo\Foo.exe from the 'File > Open Executable' menu, I see this error:
*** ERROR: Symbol file could not be found. Defaulted to export symbols for ntdll.dll -
Foo.pdb is not present in the same folder where Foo.exe is present. Why do I not see an error for Foo.exe?
On running the command ld * in WinDbg, I see these errors:
0:000> ld *
*** ERROR: Module load completed but symbols could not be loaded for Foo.exe
Symbols loaded for Foo
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\system32\msvcrt.dll -
Symbols loaded for msvcrt
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\system32\kernel32.dll -
Symbols loaded for kernel32
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\WinSxS\x86_Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_8.0.50727.6195_x-ww_44262B86\MSVCR80.dll -
Symbols loaded for MSVCR80
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\WinSxS\x86_Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_8.0.50727.6195_x-ww_44262B86\MSVCP80.dll -
Symbols loaded for MSVCP80
Symbols already loaded for ntdll
Now, let me show two attempts to fix it. One worked, the other didn't.
Downloading Windows Server 2003 with Service Pack 2 x86 retail symbols didn't work
I went to http://msdn.microsoft.com/en-us/windows/hardware/gg463028 and downloaded:
Windows Server 2003 with Service Pack 2 x86 retail symbols, all languages (File size: 154 MB - Most customers want this package.)
I ran it and installed the symbols in C:\Symbols. In WinDbg, I set the symbol path as:
C:\Symbols;C:\MySymbols
C:\MySymbols contained the symbol for Foo.exe which had a file-name: Foo.pdb.
On opening C:\foo\Foo.exe from the 'File > Open Executable' menu, I got the same error mentioned in the previous section.
However, the ld * command showed fewer errors this time.
0:000> ld *
Symbols loaded for Foo
Symbols loaded for msvcrt
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\system32\kernel32.dll -
Symbols loaded for kernel32
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\WinSxS\x86_Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_8.0.50727.6195_x-ww_44262B86\MSVCR80.dll -
Symbols loaded for MSVCR80
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\WinSxS\x86_Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_8.0.50727.6195_x-ww_44262B86\MSVCP80.dll -
Symbols loaded for MSVCP80
Symbols already loaded for ntdll
Using Microsoft Symbol Server fixed it
This time I set the symbol path as:
SRV*C:\MicrosoftSymbols*http://msdl.microsoft.com/download/symbols;C:\MySymbols
Now, on opening C:\foo\Foo.exe from the 'File > Open Executable' menu, I got no errors. I found that C:\microsoftsymbols\ntdll.pdb\F7024C7F15FE4BEA992FF38BE58AC11C2\ntdll.pdb was downloaded automatically.
This time ld * command also ran fine without any errors.
0:000> ld *
Symbols loaded for Foo
Symbols loaded for msvcrt
Symbols loaded for kernel32
Symbols loaded for MSVCR80
Symbols loaded for MSVCP80
Symbols already loaded for ntdll
The following additional symbols were downloaded automatically while the command was running.
C:\microsoftsymbols\kernel32.pdb\BE496DC9472F4438B080C70594D8F9CC2\kernel32.pdb
C:\microsoftsymbols\msvcp80.i386.pdb\E8A9423E890A4ADDB38F8F756268437C1\msvcp80.i386.pdb
C:\microsoftsymbols\msvcr80.i386.pdb\54C9E2F351544D1CB39517DC4B299EA81\msvcr80.i386.pdb
C:\microsoftsymbols\msvcrt.pdb\A7F38CEE7E684B94B7AA9FFFCAB446851\msvcrt.pdb
Questions
Why did I not see an error regarding symbol file for Foo.exe when no symbol path was specified?
Why downloading Windows Server 2003 with Service Pack 2 x86 retail symbols and specifying the path to it as the symbol path didn't work?
More information
In case it helps you to answer my question, here is my system details as obtained from winmsd.exe > System Summary:
OS Name: Microsoft(R) Windows(R) Server 2003, Enterprise Edition
Version: 5.2.3790 Service Pack 2 Build 3790
Other OS Description : Not Available
OS Manufacturer: Microsoft Corporation
System Name: EULER
System Manufacturer: VMware, Inc.
System Model: VMware Virtual Platform
System Type: X86-based PC
Processor: x86 Family 16 Model 4 Stepping 2 AuthenticAMD ~2417 Mhz
Processor: x86 Family 16 Model 4 Stepping 2 AuthenticAMD ~2416 Mhz
BIOS Version/Date: Phoenix Technologies LTD 6.00, 7/22/2008
SMBIOS Version: 2.4
Windows Directory: C:\WINDOWS
System Directory: C:\WINDOWS\system32
Boot Device: \Device\HarddiskVolume1
Locale: United States
Hardware Abstraction Layer: Version = "5.2.3790.3959 (srv03_sp2_rtm.070216-1710)"
User Name: Not Available
Time Zone: India Standard Time
Total Physical Memory: 3,839.45 MB
Available Physical Memory: 1.56 GB
Total Virtual Memory: 5.60 GB
Available Virtual Memory: 3.33 GB
Page File Space: 2.00 GB
Page File: C:\pagefile.sys
Why did I not see an error regarding symbol file for Foo.exe when no symbol path was specified?
Answer to 1:
You did when windbg tried to load the symbols for Foo
0:000> ld *
*** ERROR: Module load completed but symbols could not be loaded for Foo.exe
Symbols loaded for Foo
The reason that it didn't show up when you first loaded was that windbg didn't try to load the symbols at first for foo. Most likely because Foo had yet to be called. The program was just starting up and your program logic hadn't actually been called yet.
Why downloading Windows Server 2003 with Service Pack 2 x86 retail symbols and specifying the path to it as the symbol path didn't work?
Answer to 2:
I can't say with 100% certainty, but most likely security patches had been applied so that the pdbs that were included with the retail symbols didn't include the changes. When you went to MS site you can get all the symbols for all released versions of the DLLs including patched versions.
Why did I not see an error regarding symbol file for Foo.exe when no
symbol path was specified?
While the previous answer is correct, note that image files contain a reference the build location of their PDBs by default. Thus, it's possible for WinDBG to magically find the PDB for an image that you built on your local machine. This is explained in more detail here:
http://analyze-v.com/?p=245
I have a native release dll that is built with symbols. There is a post build step that modifies the dll. The post build step does some compression and probably appends some data. The pdb file is still valid however neither WinDbg nor Visual Studio 2008 will load the symbols for the dll after the post build step. What bits in either the pdb file or the dll do we need to modify to get either WinDbg or Visual Studio to load the symbols when it loads a dump in which our release dll is referenced?
Is it filesize that matters? A checksum or hash? A timestamp?
Modify the dump? or modify the pdb? modify the dll before it is shipped?
(We know the pdb is valid because we are able to use it to manually get symbol names for addresses in dump callstacks that reference the released dll. It's just a total pain in the *ss do it by hand for every address in a callstack in all the threads.)
This post led me to chkmatch. On the processed dll, chkmatch shows this info:
Executable:
TimeDateStamp: 4a086937
Debug info: 2 ( CodeView )
TimeStamp: 4a086937 Characteristics: 0 MajorVer: 0 MinorVer: 0
Size: 123 RVA: 00380460 FileOffset: 00380460
CodeView signature: sUar
Debug information file:
Format: PDB 7.00
Result: unmatched (reason: incompatible debug information formats)
With the same pdb against the pre-processed dll, it reports this:
Executable:
TimeDateStamp: 4a086937
Debug info: 2 ( CodeView )
TimeStamp: 4a086937 Characteristics: 0 MajorVer: 0 MinorVer: 0
Size: 123 RVA: 00380460 FileOffset: 00380460
CodeView format: RSDS
Signature: (my guid) Age: 19
PdbFile: (my path)
Debug information file:
Format: PDB 7.00
Signature: (my matching guid) Age: 19
I opened up both versions of the dll and went to offset 00380460. In the original version, clear enough I see the name of the pdb, but in the post-processed version there is no pdb info at that offset. I searched for the pdb path and found the exact same block - just at a different offset. Then I did bin search for the bytes "38 00 60 04" in the original dll. Looking at the same offset in the processed dll, I found the same bytes. So I adjusted the RVA and the offset (located by matching the bytes). Bingo! Now chkmatch reports the exact same results for the processed dll as the original (aside from the RVA and FileOffset that I changed).
Edit: Confirmed, now Visual Studio loads the symbols for dumps that reference the processed dll.
in Windbg try using the .symopt +40, this will force loading the pdb.