I'm trying to debug some code coming from the Crypto++ library, but I'm getting non-sensical information during the session.
The function of interest is DEREncodePrivateKey. Its a member function on DL_PrivateKey_EC<T> (Crypto++ is heavily templated).
228 pk.DEREncodePrivateKey(encoder);
(gdb) s
non-virtual thunk to CryptoPP::DL_PrivateKey_EC<CryptoPP::ECP>::DEREncodePrivateKey(CryptoPP::BufferedTransformation&) const
(this=0x7fff5fbfeca0, bt=#0x7fff5fbfe540) at dll.cpp:690
Line number 690 out of range; dll.cpp has 146 lines.
dll.cpp can be found at trunk / c5 / dll.cpp, and it only has 146 lines ad gdb reported.
The object was dynamic_cast just before the line in question:
const PKCS8PrivateKey& pk = dynamic_cast<const PKCS8PrivateKey&>(key);
I built the library from sources with -O0 -g3, so I think I minimized some/ most of the typical problems.
I've tried building the library and my test programs with different compilers (g++ and clang++), and I've tried debugging it with different debuggers (gdb and lldb). But I still get the non-sensical information and the library cannot be stepped in this area. Other areas are OK (as can be seen before the issue).
I'm also certain that I'm using my version of the library. Its being linked as a static lib using the full path to the library, and info shared confirms Apple is not sneaking in a dynamic library.
I need to step DL_PrivateKey_EC<CryptoPP::ECP>::DEREncodePrivateKey to see what's going on. I think the function that's being called is in eccrypto, but I'd like to see it under the debugger.
Any ideas how to proceed?
It sounds like the line table in the debug info is incorrect. On Mac OS X you can dump the line table with dwarfdump --debug-line appname.dSYM or if you are looking at a specific object file, dwarfdump --debug-line dll.o. From within lldb you can also do target modules dump line-table dll.cpp. My guess is that some function from a header file was inlined into your dll.cpp -- from line 690 -- but the linetable may incorrectly forget to switch from dll.cpp to your header file.
It seems unlikely that you would see the same problem from both g++ and from clang++. I suspect at least part of your project was not rebuilt.
fwiw you can continue debugging here -- the only problem is that the debugger won't be showing you the source as you step through your method. But you can continue to step and I expect you'll be back in your dll.cpp sources again in no time.
Related
TL;DR: Debuigging into MFC (CString) header code does not work on both my machines and as far as I can tell this is due to the peculiar way these headers are compiled.
Stepping through MFC header code when entered via disassembly works, but setting brealpoints does not work.
I'm looking for a workaround or at least acknowledgement of my analysis.
System:
Visual Studio 2019 Professional 16.9.6
Windows 10 / 1809 Enterprise LTSC
Setup: (I do apologize for this being rather long.)
Create a Visual Studio 2019 Example MFC Application Project (SDI App)
Make sure Enable Just My Codeis off under Options -> Debugging -> General.
Set the build configuration to Debug/x64 (does not make a difference, but let's all stay on the same page)
Navigate to MFCApplication1.cpp -> CMFCApplication1App::InitInstance()
Insert a CString init like this:
...
InitCommonControlsEx(&InitCtrls);
CWinAppEx::InitInstance(); // please put breakpoint 1 here
// Add this line and set breakpoints
CString this_is_text(L"Debugging into CString Header does not work!"); // breakpoint 2 here
Now, you can start the program under the debugger, and you should stop at the first breakpoint:
Now, make sure all symbols are loaded, easiest done via the Call Stack:
Just select all lines in the call stack window and hit Load Symbols in the context menu. Afterwards the call stack should look roughly like this:
> MFCApplication1.exe!CMFCApplication1App::InitInstance() Line 75 C++
mfc140ud.dll!AfxWinMain(HINSTANCE__ * hInstance=0x00007ff7b5070000, ...) Line 37 C++
MFCApplication1.exe!wWinMain(HINSTANCE__ * hInstance=0x00007ff7b5070000, ...) Line 26 C++
MFCApplication1.exe!invoke_main() Line 123 C++
MFCApplication1.exe!__scrt_common_main_seh() Line 288 C++
MFCApplication1.exe!__scrt_common_main() Line 331 C++
MFCApplication1.exe!wWinMainCRTStartup(void * __formal=0x000000c2b7084000) Line 17 C++
kernel32.dll!BaseThreadInitThunk() Unknown
ntdll.dll!RtlUserThreadStart() Unknown
Now, you can try stepping-into (possibly F11) the CWinAppEx::InitInstance() function, which should work without a problem, landing you in mfc140ud.dll!CWinApp::InitInstance() Line 394 - this is OK.
Step out again, and then then try to step-into the CString ctor:
This DOES NOT work on my machine(s)!
What I can do however, is (from the point above) switch to disassembly view, step into the calls there and get into the header code this way:
I can then successfully step through (but never into) the MFC header code. Trying to set a breakpoint will result in the error:
The breakpoint will not currently be hit. No executable code of the debugger's code type is associated with this line.
Possible causes include ...
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.28.29910\atlmfc\include\cstringt.h
And this is where I'm at.
Analysis:
What we can see from the MFC code is that we can step into "regular" cpp code, but as soon as we try to step into (or set breakpoint) code that is inside this CStringt.h it breaks.
Peculiar here: This is template header code, and still the executed code (as shown by the disassembly) is not in the user module but in the mfc###.dll! I think they do some clever tricks with the preprocessor (see defined(_MFC_DLL_BLD) and somesuch) which enables this multi use of the header file, and maybe, possibly this is also what breaks the debugger.
Question:
Is this a known problem, does this happen with all VS2019 installs, is there something peculiar to my setup?
Maybe fixed in a newer VS version?
Iff this is actually broken, what would be a useable workaround, other than constantly switching to disassembly view when into the MFC headers.
The most interesting answer here would actually be as to WHY this breaks - where does the debugger get confused? Is this a general problem with re-define-ing code when debugging library code?
The source shipped with MSVC does not match.
I think this happen, as DLLs got updated with Windows Update or a new vcredist, but Visual Studio includes are not updated. If you build with /MT or /MTd and link MFC statically, the problem does not persist.
Probably this can be reported to http://developercommunity.visualstudio.com if you care.
Workaround 1
Do steps described by #selbie:
Set a breakpoint on the line of code I want to step into.
When
the breakpoint is reached, right click in the editor window and select
"Go To Disassemly".
In disassembly mode, step over until you get to
a call statement. [...] You
can flip out of disassembly mode by right-clicking again and selecting
"go to source code".
(skipped the part not relevant to this issue)
Then pick up the location of the header manually, the debugger will tell that it does not match. The difference seem to be insignificant though, so the header is usable.
Workaround 2
Link MFC statically, compile with /MT or /MTd
Workaround 3
ATL has a similar CString that does not suffer from the issue:
#include <atlbase.h>
#include <atlstr.h>
int main() {
ATL::CString this_is_text("Debugging into CString header works");
}
Analysis went sideways at some point, but we finally found one part of the problem here:
The Require source files to exactly match the original version option:
was the problem, but in a very peculiar way:
When you do NOT require source files to match (that is, disable this default option), then the erroneous behavior of the OP occurs: The debugger can no longer match the symbols to the cstringt.h file.
Unfortunately, I had this disabled on both machines. Pulling in a third machine showed that we could set breakpoints (though F11 still does not work) and by comparing the xml export of the VS settings we found that this was different.
So, long story short: For us, to be able to set breakpoints in the (unmodified!) MFC header, requires us to enable the Require source files to exactly match .. option.
If the option is disabled, which would imply a more lenient behavior by the debugger, it no longer works.
And, yes, we double checked it's always the same source file at C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.28.29910\atlmfc\include\cstringt.h
The mystery with step-into/F11 persists, but I guess this would better be taken to a separate question.
Uncheck the Enable Just My Code option in Tools->Options->Debugging
I know that works for c++ std:: library code debugging. The other technique I do, when I forget to uncheck this option, is similar to what you describe above.
Set a breakpoint on the line of code I want to step into.
When the breakpoint is reached, right click in the editor window and select "Go To Disassemly".
In disassembly mode, step over until you get to a call statement. That's typically the std library. Eventually, you'll navigate into a mix of assembly and system code sources. You can flip out of disassembly mode by right-clicking again and selecting "go to source code".
I have been running into trouble recently trying to symbolicate a crash log of an iOS app. For some reason the UUID of the dSYM was not indexed in Spotlight. After some manual search and a healthy dose of command line incantations, I managed to symbolicate partially the crash log.
At first I thought the dSYM might be incomplete or something like that, but then I realized that the method calls missing were the ones occurring in C++ code: this project is an Objective-C app that calls into C++ libraries (via Objective-C++) which call back to Objective-C code (again, via Objective-C++ code). The calls that I'm missing are, specifically, the ones that happen in C++ land.
So, my question is: is there some way that the symbolication process can resolve the function calls of C++ code? Which special options do I need to set, if any?
One useful program that comes with the apple sdk is atos (address to symbol). Basically, here's what you want to do:
atos -o myExecutable -arch armv7 0x(address here)
It should print out the name of the symbol at that address.
I'm not well versed in Objective-C, but I'd make sure that the C++ code is being compiled with symbols. Particularly, did you make sure to include -rdynamic and/or -g when compiling the C++ code?
try
dwarfdump --lookup=0xYOUR_ADRESS YOUR_DSYM_FILE
you will have to look up each adress manually ( or write a script to do this ) but if the symbols are ok ( your dSym file is bigger than say 20MB) this will do the job .
I have Core static library, a few Component static libraries that relays on the Core one, and then there is an App that links against both Core and Component libraries. My App can link both against Core and Component as long as Component don't uses classes from Core (App uses classes from Core).
I got the following error in both armv6 and armv7 versions. So my problem is not the very popular linking issue that everyone has.
ld: symbol(s) not found for architecture armv6
clang: error: linker command failed with exit code 1 (use -v to see invocation)
I added reference to Core in Component and even added it in "Link Binary With Libraries" which shouldn't be necessary for static lib.
Since I start having this issue I start doubting my design... It probably makes more sense in dynamically linking environment but still it should be doable in static one, especially since this already works under Windows with MSVC compilers.
Edit:
I made some progress! Although I still don't know where to go with it.
Here is my setup:
Core has a class cResourceManager that has a templated method GetResource<T>(int id)
Core also has class cResource
Component has class cMesh that inherits cResource
Here are some tests:
If I try from App to call rm->GetResource<cMesh>(...) I get the linking error
If I try from App to construct cMesh I get linking the linking error
If I try from App to call static method that will return new instance of cMesh I get the linking error
If I comment out the construction of cMesh but leave other member cMesh function calls the App links fine. I can even call delete mesh.
I have never seen anything like it!
If you remove the cMesh constructor, then you are then using the default (no argument, no body) cMesh constructor that is given to you. It almost sounds like there's a build error or missing code as a result of some code in your cMesh constructor and so the library isn't actually getting generated, and perhaps Xcode isn't reporting the error. Xcode is no good at reporting linker errors.
I would suggest looking at what symbols the linker says are missing and double-check that they are actually defined in your code. My guess is that you're using one of those symbols in your cMesh constructor. A lot of times with virtual base classes, you may forget to define and implement a method or two in a child class. Could be a result of missing a method based on your template, or your template isn't #included correctly. This could compile fine but result in linker errors like you're seeing.
If Xcode isn't showing you the full linker error, show the Log Navigator (Command ⌘+7), double-click the last "Build " entry, select the error, and then press the button on the far-right of the row that appears when selected. The symbols should be listed there. If not, it's time for xcodebuild in the Terminal.
If it's not that case, I'd be interested in seeing the results of whether or not the library is being built for the appropriate architecture, or maybe this can spur some progress:
In the Xcode Organizer Shift ⇧+Command ⌘+2, click Projects and find the path to the DerivedData for your project.
In the Terminal, navigate to that directory (cd ~/Library/Developer/Xcode/DerivedData/proj-<random value>/)
Remove (or move aside) the Build directory (rm -r Build)
In Xcode, try to build with the cMesh constructor present.
Find the Library product file (cd Build/Products/<scheme>-iphoneos)
Your compiled static libraries (<libname>.a) should be in this directory. If they're not there, they didn't build (unless you put your products elsewhere). If your libraries are there, let's confirm that they actually are getting built for the appropriate architecture. Run otool -vh <library>.a. You should see something like:
$ otool -vh libtesting.a
Archive : libtesting.a
libtesting.a(testing.o):
Mach header
magic cputype cpusubtype caps filetype ncmds sizeofcmds flags
MH_MAGIC ARM V7 0x00 OBJECT 3 1928 SUBSECTIONS_VIA_SYMBOLS
As you can see, my test library was built for ARMv7.
Make sure you are linking them in the correct order.
If Component depends on symbols in Core, then Component needs to be first in the link order, so the linker knows which symbols to look for in Core.
In MSVC the order doesn't matter, but in most other compiler suites it does.
I don't think Clang generates code for armv6, if you're targeting devices that old you still need to use GCC.
I am currently writing a small library, and I want to check it for leaks (among other things); however, for some reason, gdb is not loading the library symbols. I have read many other posts on here (and various other places on the internet) about this, however, I cannot seem to find a solution. Here is what is going on:
I am compiling the shared library with the following flags (these are included in both the final shared library as well as all object files):
CFLAGS=-Wall -O0 -g -fPIC
Likewise, I am compiling the binary memtest (the client application for the library) to check for memory leaks and such with these flags
CFLAGS=-Wall -O0 -g
Now, I inserted a NULL pointer into the library to test if I could trace through it and "debug" the pointer (i.e. it's making it crash). So I try to run it through gdb, but it's a no go. The output of info sharedlibrary is the same for both the executable and the core:
(gdb) info sharedlibrary
From To Syms Read Shared Object Library
... Some libraries I am not worried about debugging...
0x00d37340 0x00d423a4 Yes (*) /home/raged/MyLIB/memtest/../lib/libMyLIB.so.0 <--- My lib
.... and some more....
(*): Shared library is missing debugging information.
As you can see, it's not loading the debug information. I am uncertain as to why this is. I have built and linked everything with the -g flag, and I even try -ggdb and -g3 but nothing seems to work properly. When I load in a core dump, this is what I see:
...some libs...
Reading symbols from /home/raged/MyLIB/memtest/../lib/libMyLIB.so.0...done.
Loaded symbols for /home/raged/MyLIB/memtest/../lib/libMyLIB.so.0
Reading symbols from /usr/lib/libstdc++.so.6...(no debugging symbols found)...done.
...some more libs...
Notice how my library does not give a (no debugging symbols found) error - anyone have any ideas why? As I said before, I am unable to debug this through running the program gdb ./memtest or through debugging the core file.
Thanks for your help.
EDIT It may also be important to note, that (if you didn't realize by path) this library is a local shared library (i.e. I'm using -Wl,-rpath to link/load it)
EDIT2 It seems my version of GDB was out-of-date. Now, I have updated to the latest version from the CVS server (I have also tried latest release version 7.2) and it can "load" symbols. My info sharedlibrary now reads this:
0x00e418b0 0x00e4be74 Yes /home/raged/MyLIB/memtest/../lib/libMyLIB.so.0
However, I am still unable to step through any functions (in the shared library) - anyone have any ideas?
EDIT3 I have also tried to step through linking against a static library (libMyLIB.a) but it still isn't working. My OS is CentOS 5.6; does anyone know of any issues with this system? Also, just another confirmation that my symbols are being loaded (it just can't step through any shared lib function for some reason)
(gdb) sharedlibrary MyLIB
Symbols already loaded for /home/raged/MyLIB/memtest/../lib/libMyLIB.so.0
I found the reason this wasn't working: I was calling an old function call to initialize a pointer in my test executable. Since the object was never being created, I could never step into the library. Once I updated the function call, all worked well.
That said, if anyone experiences similar issues while all symbols appear to be loaded, be sure to check that all pointers are initialized properly even if they have the correct type.
I am using g++ 4.1.2 and gdb 7.2
I am debugging code that uses Xerces, which I built using the same tools, though presumably without debugging.
GDB steps through my code just fine, but of course does NOT step through Xerces because it probably doesn't have debugging information, and definitely does not know where the source directory is. But all I want is to set a breakpoint when Xerces (a callback parser) calls a callback object.
Their base class is DefaultHandler
I have a class ContentHandlerBase : public DefaultHandler
Then leaf classes inherit from ContentHandlerBase. These leaf classes are inside namespace A, for example
in gdb I try to set a breakpoint.
b A::LeafContentHandler::LeafContentHandler
b A::LeafContentHandler::endElement
The first breakpoint works because the code is inline (defined in the header).
The second breakpoint does not work, meaning gdb claims that no such symbol exists, even though it obviously does. It is a virtual function defined in the Xerces library, if that makes a difference. Before I recompiled Xerces, it was built with g++3.4.6 and I could not find the symbol in gdb. Now, gdb finds the symbol (I can hit tab) but then it says it doesn't exist, should I wait for a library to load.
Can anyone tell me what I have to do to make it work? I'd prefer not to build all of xerces with debugging.
Note that in some cases, with the constructor in the .cpp file, it also worked for some reason, and then, because it was in the same file, I could set a subsequent breakpoint at linenumber, and that worked.
Try gdb 7.1 - it seems there are some problems in setting breakpoint by function name in gdb 7.2