Symbols (pdb) for native dll are not loaded due to post build step - c++

I have a native release dll that is built with symbols. There is a post build step that modifies the dll. The post build step does some compression and probably appends some data. The pdb file is still valid however neither WinDbg nor Visual Studio 2008 will load the symbols for the dll after the post build step. What bits in either the pdb file or the dll do we need to modify to get either WinDbg or Visual Studio to load the symbols when it loads a dump in which our release dll is referenced?
Is it filesize that matters? A checksum or hash? A timestamp?
Modify the dump? or modify the pdb? modify the dll before it is shipped?
(We know the pdb is valid because we are able to use it to manually get symbol names for addresses in dump callstacks that reference the released dll. It's just a total pain in the *ss do it by hand for every address in a callstack in all the threads.)

This post led me to chkmatch. On the processed dll, chkmatch shows this info:
Executable:
TimeDateStamp: 4a086937
Debug info: 2 ( CodeView )
TimeStamp: 4a086937 Characteristics: 0 MajorVer: 0 MinorVer: 0
Size: 123 RVA: 00380460 FileOffset: 00380460
CodeView signature: sUar
Debug information file:
Format: PDB 7.00
Result: unmatched (reason: incompatible debug information formats)
With the same pdb against the pre-processed dll, it reports this:
Executable:
TimeDateStamp: 4a086937
Debug info: 2 ( CodeView )
TimeStamp: 4a086937 Characteristics: 0 MajorVer: 0 MinorVer: 0
Size: 123 RVA: 00380460 FileOffset: 00380460
CodeView format: RSDS
Signature: (my guid) Age: 19
PdbFile: (my path)
Debug information file:
Format: PDB 7.00
Signature: (my matching guid) Age: 19
I opened up both versions of the dll and went to offset 00380460. In the original version, clear enough I see the name of the pdb, but in the post-processed version there is no pdb info at that offset. I searched for the pdb path and found the exact same block - just at a different offset. Then I did bin search for the bytes "38 00 60 04" in the original dll. Looking at the same offset in the processed dll, I found the same bytes. So I adjusted the RVA and the offset (located by matching the bytes). Bingo! Now chkmatch reports the exact same results for the processed dll as the original (aside from the RVA and FileOffset that I changed).
Edit: Confirmed, now Visual Studio loads the symbols for dumps that reference the processed dll.

in Windbg try using the .symopt +40, this will force loading the pdb.

Related

I can not understand symbol files functioning properly

I'm studying kernel mode driver following to this Youtube video and preparing for debugging a driver in a VirtualBox VM, with WinDbg and Virtual KD.
I set up the symbol file by clicking
File / Symbol file path
add symbol path
SRV*c:\symbols* http://msdl.microsoft.com/download/symbols
put a check mark to "reload" item
click ok
After that, Windbg's screen is as follows:
************* Path validation summary **************
Response Time (ms) Location
Deferred SRV*c:\symbols* http://msdl.microsoft.com/download/symbols
kd> .reload
Connected to Windows 10 17134 x64 target at (Sun Oct 7 13:16:30.147 2018 (UTC + 9:00)), ptr64 TRUE
Loading Kernel Symbols
...............................................................
................................................................
..........................
Loading User Symbols
Loading unloaded module list
......Unable to enumerate user-mode unloaded modules, Win32 error 0n30
I can not understand symbol files functioning properly.
Are Symbol files currently not available?
I use lml command in such case.
If symbol files are loaded, you can find module name like this.
2: kd> lml
start end module name
ffff9e54`ba960000 ffff9e54`ba9d7000 win32k (pdb symbols) c:\symbols\win32k.pdb\901A464ABCFD2696F50FFB02C607B4661\win32k.pdb
fffff803`6921a000 fffff803`69aef000 nt (pdb symbols) c:\symbols\ntkrnlmp.pdb\9378084E8DBD4AB1A155099BCE693E341\ntkrnlmp.pdb

nsi.pdb Cannot Load Symbol Error When I am trying to toggle breakpoint C++

I got an error when I try to put a breakpoint to a line. It puts correctly but when I start the program it says This BP will not be currenlt hit. Because no symbols loaded.
When I take a look at modules list I saw NSI.pdb doesnt exist or couldnt be loaded Im not sure why?
I installed Debug tools for windows 86 - 64 both.
here's the modules windows

Debug Crash in DLL

Trying to debug a crash in one of our DLL's. It is loaded into Server Manager and crashes when trying to configure Active Directory Certificate Services (the DLL is a registered provider). I know the crash is an access violation and I have the pdb file, just don't know how to go about debugging this. I've read pages such as this and this (didn't help). I tried to glean the info using windbg (using lm to get the loaded address, which appears to be 8000000:
"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\windbg.exe" -z myKSP.dll
Then
0:000> lm
start end module name
00000001`80000000 00000001`8005e000 ...
Then, since the Event Viewer tells me:
Exception code: 0xc0000005
Fault offset: 0x000000000002a601
I tried to view that:
0:000> ln 80000000+2a601
Browse module
Set bu breakpoint
Nothing is shown.
I have VS2015, so, I tried to attach to the serververmanager.exe process. Next, I tried loading symbols via Tools->Options->Debugging->Symbols and specifying the path, but, when I set a breakpoint, I always receive "no symbols have been loaded". In the previous symbol windows, I set the cache folder, which downloaded a bunch of stuff, but that did not seem to load anything.
Clearly, I'm not using the tools correctly. How do I debug a DLL, compiled in Release mode, PDB is available, that is loaded by the ServerManager.exe or whatever sub-process it might spawn)?
Start windbg, press Ctrl-D to open your dump file, then type the following. That should give you either a significant stack after one of the kp500, or at least will tell you whether the pdb file doesn't match the binary.
.symfix
.sympath+ <FOLDER_WITH_YOUR_PDB>
.reload
!sym noisy
.reload /v /f myKSP.dll
!sym quiet
kp500
.ecxr
kp500

How to get list of functions inside a DLL (managed and unmanaged)?

So I play around a DLL (UnityEditor.dll) I wanna to get a list of all functions inside of this managed DLL which are unmanaged (dll is probably composed from a native C++ (with statically compiled libaries if such were used) core and managed C++ wrapper all wrapped into one dll.) I want to get a list of all unmanaged functions inside of that Dll to for example create my own managed\unmanaged wrapper?
The dumpbin.exe utility shipped with Visual Studio can be used to display a list of exports. For example:
dumpbin.exe /EXPORTS C:\WINDOWS\System32\Kernel32.dll
Example output:
Microsoft (R) COFF/PE Dumper Version 10.00.30319.01
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file C:\Windows\System32\kernel32.dll
File Type: DLL
Section contains the following exports for KERNEL32.dll
00000000 characteristics
4E20FBA0 time date stamp Sat Jul 16 03:46:56 2011
0.00 version
1 ordinal base
1390 number of functions
1390 number of names
ordinal hint RVA name
1 0 AcquireSRWLockExclusive (forwarded to NTDLL.RtlAcquireSRWLockExclusive)
2 1 AcquireSRWLockShared (forwarded to NTDLL.RtlAcquireSRWLockShared)
3 2 00004440 ActivateActCtx
4 3 00066B80 AddAtomA
5 4 00066B20 AddAtomW
6 5 0006ADF0 AddConsoleAliasA
7 6 0006AE60 AddConsoleAliasW
Open the .dll file and look for the EXPORT section of this PE file using the binary PE/COFF specs available from Microsoft.
But that's an overkill, I think. Your question should be a concrete want. What exactly do you want to wrap and what do you have ? Only the binaries and no source/headers ?
DLLs don't contain "functions". They contain code and entrypoints. It's not possible to tell from optimized code where transitions between functions occur unless you have a debug database.

Why WinDbg is unable to load symbols from Windows Server 2003 with Service Pack 2 x86 retail symbols download?

I wrote a C++ project called 'Foo' using Microsoft Visual Studio 2005 Verison 8.0.50727.762 (SP.050727-7600) on Windows XP Professional Version 2002 Service Pack 3. I built the project into Foo.exe. Then, I copied the file Foo.exe to a Windows Server 2003 Enterprise Edition Service Pack 2. It runs fine.
Now, I try to debug it using WinDbg:6.12.0002.663 X86 Version 5.2.
No symbol path, source path or image path is set
When I open C:\foo\Foo.exe from the 'File > Open Executable' menu, I see this error:
*** ERROR: Symbol file could not be found. Defaulted to export symbols for ntdll.dll -
Foo.pdb is not present in the same folder where Foo.exe is present. Why do I not see an error for Foo.exe?
On running the command ld * in WinDbg, I see these errors:
0:000> ld *
*** ERROR: Module load completed but symbols could not be loaded for Foo.exe
Symbols loaded for Foo
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\system32\msvcrt.dll -
Symbols loaded for msvcrt
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\system32\kernel32.dll -
Symbols loaded for kernel32
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\WinSxS\x86_Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_8.0.50727.6195_x-ww_44262B86\MSVCR80.dll -
Symbols loaded for MSVCR80
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\WinSxS\x86_Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_8.0.50727.6195_x-ww_44262B86\MSVCP80.dll -
Symbols loaded for MSVCP80
Symbols already loaded for ntdll
Now, let me show two attempts to fix it. One worked, the other didn't.
Downloading Windows Server 2003 with Service Pack 2 x86 retail symbols didn't work
I went to http://msdn.microsoft.com/en-us/windows/hardware/gg463028 and downloaded:
Windows Server 2003 with Service Pack 2 x86 retail symbols, all languages (File size: 154 MB - Most customers want this package.)
I ran it and installed the symbols in C:\Symbols. In WinDbg, I set the symbol path as:
C:\Symbols;C:\MySymbols
C:\MySymbols contained the symbol for Foo.exe which had a file-name: Foo.pdb.
On opening C:\foo\Foo.exe from the 'File > Open Executable' menu, I got the same error mentioned in the previous section.
However, the ld * command showed fewer errors this time.
0:000> ld *
Symbols loaded for Foo
Symbols loaded for msvcrt
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\system32\kernel32.dll -
Symbols loaded for kernel32
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\WinSxS\x86_Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_8.0.50727.6195_x-ww_44262B86\MSVCR80.dll -
Symbols loaded for MSVCR80
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\WinSxS\x86_Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_8.0.50727.6195_x-ww_44262B86\MSVCP80.dll -
Symbols loaded for MSVCP80
Symbols already loaded for ntdll
Using Microsoft Symbol Server fixed it
This time I set the symbol path as:
SRV*C:\MicrosoftSymbols*http://msdl.microsoft.com/download/symbols;C:\MySymbols
Now, on opening C:\foo\Foo.exe from the 'File > Open Executable' menu, I got no errors. I found that C:\microsoftsymbols\ntdll.pdb\F7024C7F15FE4BEA992FF38BE58AC11C2\ntdll.pdb was downloaded automatically.
This time ld * command also ran fine without any errors.
0:000> ld *
Symbols loaded for Foo
Symbols loaded for msvcrt
Symbols loaded for kernel32
Symbols loaded for MSVCR80
Symbols loaded for MSVCP80
Symbols already loaded for ntdll
The following additional symbols were downloaded automatically while the command was running.
C:\microsoftsymbols\kernel32.pdb\BE496DC9472F4438B080C70594D8F9CC2\kernel32.pdb
C:\microsoftsymbols\msvcp80.i386.pdb\E8A9423E890A4ADDB38F8F756268437C1\msvcp80.i386.pdb
C:\microsoftsymbols\msvcr80.i386.pdb\54C9E2F351544D1CB39517DC4B299EA81\msvcr80.i386.pdb
C:\microsoftsymbols\msvcrt.pdb\A7F38CEE7E684B94B7AA9FFFCAB446851\msvcrt.pdb
Questions
Why did I not see an error regarding symbol file for Foo.exe when no symbol path was specified?
Why downloading Windows Server 2003 with Service Pack 2 x86 retail symbols and specifying the path to it as the symbol path didn't work?
More information
In case it helps you to answer my question, here is my system details as obtained from winmsd.exe > System Summary:
OS Name: Microsoft(R) Windows(R) Server 2003, Enterprise Edition
Version: 5.2.3790 Service Pack 2 Build 3790
Other OS Description : Not Available
OS Manufacturer: Microsoft Corporation
System Name: EULER
System Manufacturer: VMware, Inc.
System Model: VMware Virtual Platform
System Type: X86-based PC
Processor: x86 Family 16 Model 4 Stepping 2 AuthenticAMD ~2417 Mhz
Processor: x86 Family 16 Model 4 Stepping 2 AuthenticAMD ~2416 Mhz
BIOS Version/Date: Phoenix Technologies LTD 6.00, 7/22/2008
SMBIOS Version: 2.4
Windows Directory: C:\WINDOWS
System Directory: C:\WINDOWS\system32
Boot Device: \Device\HarddiskVolume1
Locale: United States
Hardware Abstraction Layer: Version = "5.2.3790.3959 (srv03_sp2_rtm.070216-1710)"
User Name: Not Available
Time Zone: India Standard Time
Total Physical Memory: 3,839.45 MB
Available Physical Memory: 1.56 GB
Total Virtual Memory: 5.60 GB
Available Virtual Memory: 3.33 GB
Page File Space: 2.00 GB
Page File: C:\pagefile.sys
Why did I not see an error regarding symbol file for Foo.exe when no symbol path was specified?
Answer to 1:
You did when windbg tried to load the symbols for Foo
0:000> ld *
*** ERROR: Module load completed but symbols could not be loaded for Foo.exe
Symbols loaded for Foo
The reason that it didn't show up when you first loaded was that windbg didn't try to load the symbols at first for foo. Most likely because Foo had yet to be called. The program was just starting up and your program logic hadn't actually been called yet.
Why downloading Windows Server 2003 with Service Pack 2 x86 retail symbols and specifying the path to it as the symbol path didn't work?
Answer to 2:
I can't say with 100% certainty, but most likely security patches had been applied so that the pdbs that were included with the retail symbols didn't include the changes. When you went to MS site you can get all the symbols for all released versions of the DLLs including patched versions.
Why did I not see an error regarding symbol file for Foo.exe when no
symbol path was specified?
While the previous answer is correct, note that image files contain a reference the build location of their PDBs by default. Thus, it's possible for WinDBG to magically find the PDB for an image that you built on your local machine. This is explained in more detail here:
http://analyze-v.com/?p=245