I found a suspicious deadlock at address myfile.exe+0x144c7 (from list of threads in ProcessExplorer). Now, I want to know which function it is.
info symbol addr
requires that addr is absolute. Is there a command that takes the relative address given by ProcessExplorer. I can add 0x400000 but it would be better if GDB could do it for me.
gdb accepts an expression for the symbol address, so you can do something like this:
info symbol 0x40000000 + 0x144c7
If you check "info variables" (or use nm on the executable) there's probably a symbolic name for the text segment containing your code, so you can also do something like:
info symbol _init + 0x144c7
Note that symbol might not work as expected if your problem is in a DLL or other text segment.
Related
Background: I'm trying to simplify my answer to the question c++ - Calling a lambda function in GDB - Stack Overflow.
Consider this simple program
int main(){
auto a=[&](int x){return x+1;};
}
When it's compiled (for example to a.out), I can see
$ nm --demangle a.out |grep lambda
000000000000113a t main::{lambda(int)#1}::operator()(int) const
$ nm a.out |grep 113a
000000000000113a t _ZZ4mainENKUliE_clEi
So there's a symbol _ZZ4mainENKUliE_clEi for the lambda operator() method in the text (code) section.
Given the exact name, I can print its value in gdb, and tab-completion also lists the symbol:
$ gdb -q ./a.out
(gdb) print _ZZ4mainENKUliE_clEi
$1 = {int (const struct {...} * const, int)} 0x55555555513a <operator()(int) const>
(gdb) print 'main::{lambda(int)#1}::operator()(int) const'
$2 = {int (const struct {...} * const, int)} 0x55555555513a <operator()(int) const>
But how can I find the symbol without using the external executable nm (inside gdb itself)?
Answers using the Python API is preferred.
Failed attempts
As suggested in c - Ask GDB to list all functions in a program - Stack Overflow:
(gdb) info functions .*lambda.*
All functions matching regular expression ".*lambda.*":
File /build/gcc/src/gcc/libstdc++-v3/src/c++11/compatibility-thread-c++0x.cc:
void std::once_flag::_Prepare_execution::_Prepare_execution<std::call_once<void (std::thread::*)(), std::
thread*>(std::once_flag&, void (std::thread::*&&)(), std::thread*&&)::{lambda()#1}>(void (std::thread::*&)())::{l
ambda()#1}::_FUN();
which is not the one I want, and
(gdb) info functions .*ZZ4.*
All functions matching regular expression ".*ZZ4.*":
doesn't match anything at all.
Also tried other things in Debugging with GDB - Examining the Symbol Table,
maintenance print symbols doesn't have it either.
In Symbols In Python (Debugging with GDB),
gdb.lookup_global_symbol("_ZZ4mainENKUliE_clEi") and gdb.lookup_static_symbol("_ZZ4mainENKUliE_clEi") both returns None.
file ./a.out in gdb does not help (suggested by GDB does not see symbols )
This solution finds the address of the .text section by parsing the textual output of info files, then find the symbol correspond to each address by parsing the output of info symbol gdb command.
Because it parses the textual output, in some special cases it might break. If anyone have any better solution/improvement please suggest/post a new answer.
I took a look at https://sourceware.org/gdb/current/onlinedocs/gdb/Progspaces-In-Python.html#Progspaces-In-Python / https://sourceware.org/gdb/current/onlinedocs/gdb/Objfiles-In-Python.html but can't find any.
I can't use block_for_pc, explained in this comment.
Run the Python code. The result is collected into the list symbols.
Read the comments in the code for more details how it works.
# (setup measure time)
import time
start=time.time()
# step 1. Find the limit of `.text`. Store in (a, b).
import re
[[a, b]]=[
(int(match[1], 16), int(match[2], 16))
for line in gdb.execute("info files", to_string=True).splitlines()
for match in [re.fullmatch(r"\s*(0x[0-9a-f]+) - (0x[0-9a-f]+) is \.text", line)]
if match
]
# Step 2. Iterate through symbols.
symbols=[]
mismatch=0
i=b-1
while i>=a:
line=gdb.execute("info symbol "+hex(i), to_string=True)
if line=='No symbol matches '+hex(i)+'.\n':
# There's no symbol here. Unfortunately I can't find any way to quickly skip through it.
i-=1
mismatch+=1
continue
# Determine the symbol and the offset.
match=re.fullmatch(r"(.+) \+ (\d+) in section \.text of .*\n?", line)
if match:
symbol=match[1]
offset=int(match[2])
else:
match=re.fullmatch(r"(.+) in section \.text of .*\n?", line)
symbol=match[1]
offset=0
# collect the symbol into the list.
i-=offset
symbols.append((symbol, hex(i)))
i-=1
# (report time taken)
print(time.time()-start, len(symbols), mismatch)
For a reasonably large executable, nm takes 0.042s, while the script takes 0.778s and report 5769 symbols and 44171 misses. (the misses takes most of the time)
I used T32 to load bin files and elf and wrote scripts to extract the Pc , Lr register values from the ELF file. Now I have the address for e.g say PC's address is 0xccccdddd. Now I need to get the symbol corresponding to that.
I ran gdb and used gdb info symbol 0xccccdddd and got the symbol name.
But I need to know if there is any command in T32 itself to get the symbol name. Or can I get the symbol name from some commands like readelf or objdump.
Thanks in advance.
The command to open a window to see all the static symbols is
sYmbol.Browse
To learn more about that window, I recommend to check the "Training HLL Debugging" (training_hll.pdf) from your TRACE32 installation.
To get only the symbol related to one single address use the PRACTICE function sYmbol.Name(<addr>). Functions have to be used together with a command. To simply display the name use the command PRINT.
E.g.:
PRINT sYmbol.Name(P:0xccccdddd)
Note that the address-offset has to be prefixed by an access class. Usually the access class "P:" stands for program memory, while "D:" stands for data memory. See the "Processor Architecture Manual" for more CPU specific access classes (Menu > Help > Processor Architecture Manual)
I am debugging a C++ application, info shared tells me this library I want to break in has been read and has debugging symbols,
0x00007fffedc1f530 0x00007ffff18e4e60 Yes /home/me/WPEWebKit/WebKitBuild/Debug/lib/libWPEWebKit.so
I then ran nm on this loaded shared library and found a symbol I would like to break on (just to make sure the name was right):
$ nm /home/me/WPEWebKit/WebKitBuild/Debug/lib/libWPEWebKit.so | c++filt | grep -iw handleSyncmessage
0000000008a03e14 T WebCore::MediaPlayerPrivateGStreamer::handleSyncMessage(_GstMessage*)
It might be worth me saying that libWPEWebKit.so is just over 650MB, so quite a heavyweight.
Now in GDB, I say this,
(gdb) break WebCore::MediaPlayerPrivateGStreamer::handleSyncMessage
Function "WebCore::MediaPlayerPrivateGStreamer::handleSyncMessage" not defined.
Make breakpoint pending on future shared library load? (y or [n])
But we know at this point the symbol is loaded. If I rerun with the breakpoint set, it never gets fired, despite having proved the function is indeed running.
Am I doing something wrong here?
Edit: Adding info functions output
(gdb) info functions handleSyncMessage
All functions matching regular expression "handleSyncMessage":
.. lots of template instantiations I am ommitting because they are from unrelated things that just ref. this type ..
File ../../Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
bool (anonymous namespace)::MediaPlayerPrivateGStreamer::handleSyncMessage(GstMessage*);
File ../../Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp:
bool (anonymous namespace)::MediaPlayerPrivateGStreamerBase::handleSyncMessage(GstMessage*);
..
Non-debugging symbols:
0x00007fffed84bcf0 WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)#plt
0x00007fffed84d810 WebCore::MediaPlayerPrivateGStreamer::handleSyncMessage(_GstMessage*)#plt
I now notice that I missed some output from nm above that I add for completeness
0000000008a03e14 T WebCore::MediaPlayerPrivateGStreamer::handleSyncMessage(_GstMessage*)
0000000008a11868 T WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)
0000000008a16fa8 t std::remove_reference<WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}&>::type&& std::move<WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}&>(std::remove_reference&&)
0000000008a16f78 t WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}& std::forward<WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}&>(std::remove_reference<WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}&>::type&)
0000000008a17959 t WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}&& std::forward<WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}>(std::remove_reference<WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}>::type&)
000000000bdac7d0 r WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::__FUNCTION__
000000000bdac800 r WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::__PRETTY_FUNCTION__
0000000008a11834 t WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#2}::operator()() const
0000000008a115a4 t WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}::operator()() const
0000000008a18128 t WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}::_GstMessage({lambda()#1}&&)
0000000008a18128 t WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}::_GstMessage({lambda()#1}&&)
0000000008a117f8 t WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}::~_GstMessage()
0000000008a117f8 t WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}::~_GstMessage()
000000000bdaddc8 r WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)::{lambda()#1}::operator()() const::__FUNCTION__
I tried breaking on the MediaPlayerPrivateGStreamerBase symbol as well as the MediaMediaPlayerPrivateGStreamer symbol to no success.
Edit 2
Another attempt,
(gdb) rbreak ^WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage
Breakpoint 1 at 0x7fffed84bcf0
<function, no debug info> WebCore::MediaPlayerPrivateGStreamerBase::handleSyncMessage(_GstMessage*)#plt;
So I guess the underlying question is why does GDB only set breaks on the procedure linkage table and not the real library function? How people even debug on Linux platforms? This seems utterly insane. Needless to say, even the breakpoint on the plt entry is not firing..
In my application I have setup signal handler to catch Segfaults, and print bactraces.
My application loads some plugins libraries, when process starts.
If my application crashes with a segfault, due to an error in the main executable binary, I can analyze the backtrace with:
addr2line -Cif -e ./myapplication 0x4...
It accurately displays the function and the source_file:line_no
However how do analyze if the crash occurs due to an error in the plugin as in the backtrace below?
/opt/myapplication(_Z7sigsegvv+0x15)[0x504245]
/lib64/libpthread.so.0[0x3f1c40f500]
/opt/myapplication/modules/myplugin.so(_ZN11ICAPSection7processEP12CONNECTION_TP7Filebufi+0x6af)[0x7f5588fe4bbf]
/opt/myapplication/modules/myplugin.so(_Z11myplugin_reqmodP12CONNECTION_TP7Filebuf+0x68)[0x7f5588fe51e8]
/opt/myapplication(_ZN10Processors7ExecuteEiP12CONNECTION_TP7Filebuf+0x5b)[0x4e584b]
/opt/myapplication(_Z15process_requestP12CONNECTION_TP7Filebuf+0x462)[0x4efa92]
/opt/myapplication(_Z14handle_requestP12CONNECTION_T+0x1c6d)[0x4d4ded]
/opt/myapplication(_Z13process_entryP12CONNECTION_T+0x240)[0x4d79c0]
/lib64/libpthread.so.0[0x3f1c407851]
/lib64/libc.so.6(clone+0x6d)[0x3f1bce890d]
Both my application and plugin libraries have been compiled with gcc and are unstripped.
My application when executed, loads the plugin.so with dlopen
Unfortunately, the crash is occurring at a site where I cannot run the application under gdb.
Googled around frantically for an answer but all sites discussing backtrace and addr2line exclude scenarios where analysis of faulty plugins may be required.
I hope some kind-hearted hack knows solution to this dilemma, and can share some insights. It would be so invaluable for fellow programmers.
Tons of thanks in advance.
Here are some hints that may help you debug this:
The address in your backtrace is an address in the address space of the process at the time it crashed. That means that, if you want to translate it into a 'physical' address relative to the start of the .text section of your library, you have to subtract the start address of the relevant section of pmap from the address in your backtrace.
Unfortunately, this means that you need a pmap of the process before it crashed. I admittedly have no idea whether loading addresses for libraries on a single system are constant if you close and rerun it (imaginably there are security features which randomize this), but it certainly isn't portable across systems, as you have noticed.
In your position, I would try:
demangling the symbol names with c++filt -n or manually. I don't have a shell right now, so here is my manual attempt: _ZN11ICAPSection7processEP12CONNECTION_TP7Filebufi is ICAPSection::process(CONNECTION_T *, Filebuf *, int). This may already be helpful. If not:
use objdump or nm (I'm pretty sure they can do that) to find the address corresponding to the mangled name, then add the offset (+0x6af as per your stacktrace) to this, then look up the resulting address with addr2line.
us2012's answer was quite the trick required to solve the problem. I am just trying to restate it here just to help any other newbie struggling with the same problem, or if somebody wishes to offer improvements.
In the backtrace it is clearly visible that the flaw exists in the code for myplugin.so. And the backtrace indicates that it exists at:
/opt/myapplication/modules/myplugin.so(_ZN11ICAPSection7processEP12CONNECTION_TP7Filebufi+0x6af)[0x7f5588fe4bbf]
The problem of locating the line corresponding to this fault cannot be determined as simplistically as:
addr2line -Cif -e /opt/myapplication/modules/myplugin.so 0x7f5588fe4bbf
The correct procedure here would be to use nm or objdump to determine the address pointing to the mangled name. (Demangling as done by us2012 is not really necessary at this point). So using:
nm -Dlan /opt/myapplication/modules/myplugin.so | grep "_ZN11ICAPSection7processEP12CONNECTION_TP7Filebufi"
I get:
0000000000008510 T _ZN11ICAPSection7processEP12CONNECTION_TP7Filebufi /usr/local/src/unstable/myapplication/sources/modules/myplugin/myplugin.cpp:518
Interesting to note here is that myplugin.cpp:518 actually points to the line where the opening "{" of the function ICAPSection::process(CONNECTION_T *, Filebuf *, int)
Next we add 0x6af to the address (revealed by the nm output above) 0000000000008510 using linux shell command
printf '0x%x\n' $(( 0x0000000000008510 + 0x6af ))
And that results in 0x8bbf
And this is the actual source_file:line_no of the faulty code, and can be precisely determined with addr2line as:
addr2line -Cif -e /opt/myapplication/modules/myplugin.so 0x8bbf
Which displays:
std::char_traits<char>::length(char const*)
/usr/include/c++/4.4/bits/char_traits.h:263
std::string::assign(char const*)
/usr/include/c++/4.4/bits/basic_string.h:970
std::string::operator=(char const*)
/usr/include/c++/4.4/bits/basic_string.h:514
??
/usr/local/src/unstable/myapplication/sources/modules/myplugin/myplugin.cpp:622
I am not too sure why the function name was not displayed here, but myplugin.cpp:622 was quite precisely where the fault was.
I was reading up on various things on CodeProject and I found this article: http://www.codeproject.com/Articles/29527/Reverse-Engineering-and-Function-Calling-by-Addres
So what I did was I created an injector and a DLL, and grabbed the sample executable file. It basically outputs this when you press F11:
http://i.stack.imgur.com/YIygV.jpg
So I followed the entire tutorial, but the thing is that the address used in the DLL is always changing. This one to be specific:
pFunctionAddress pMySecretFunction = (pFunctionAddress)(0x004113C0);
In his tutorial the address for the function is 0x004113C0. In mine it is something else, and I take the one I have and use it. It works perfectly, but when I close the executable and open it, it won't work anymore, and OllyDbg shows that the address is a totally new one.
So I researched a bit and I started adding breakpoints with OllyDbg. I found out that the address is always going to be:
main + 4C
Where I guess "main" is the main module of these executable. How can I find this address to the function always? Because it changes all the time and I am clueless at this point. In this article I read it doesn't go through what happens when the executable is re-opened, and I've spent 5 hours trying to find a solution.
Thanks in advance!
EDIT:
Huge thanks to everyone. Thanks to mfc especially, I have finally figured it out! What I ended up doing was whenever I hit DLL_PROCESS_ATTACH, I set a global HMODULE to the address of the executable, like this:
HMODULE g_hExeModule;
g_hExeModule = GetModuleHandle(L"TutExample.exe");
And after a few tests it seems like the function address is always the address of the executable + 0x11014, so in the call I just do:
pFunctionAddress pMySecretFunction = (pFunctionAddress)((DWORD)g_hExeModule + 0x11014);
so if I find a way to get the address of "main" I can add a 4C offset and the function will always be there, I think
Again, functions do have an address:
void *(funcPtr)() = (void (*)())((char *)&main + 0x4C);
// If you were right, and you also substituted the appropriate
// function signature above, then this should work:
funcPtr();
The function that you are trying to call is inside of the exe file, so the reference offset should be relative to the memory address that the exe is loaded.
The offset to the target function should be a constant, changed only after each compilation of the source code.
To find out more about your exe, add these two line into your exe:
printf(_T("Exe loaded at: %08X"), GetModuleHandle(_T("TutExample.exe")));
printf(_T("Target function at: %08X"), mySecretFunction);
I cannot edit my post nor add any remark, so I have to post this as a new answer.
Your result:
Exe loaded at: 00000000 (wrong, probably: 00BE0000 and offset is: 00001005)
Target function at: 00BE1005
Exe loaded at: 00000000 (wrong, probably: 01230000 and offset is: 00001005)
Target function at: 01231005
Exe loaded at: 00000000 (wrong, probably: 012A0000 and offset is: 00001005)
Target function at: 012A1005
Please check the name of your compiled exe, is it "TutExample.exe" ? if not, change it to the exact name in the call to GetModuleHandle.
The value of "00000000" indicates that the GetModuleHandle fails because the name "TutExample.exe" is not found in the current memory space.
The address of target function seems ok. Just minus this address with the address of loaded exe and you will get the offset inside the exe memory layout.
You can do this same math inside your injected dll to alway tracks the target function address correctly no matter how the os loads the exe.
Newer OS's have a feature called ASLR (Address space layout randomization). Criminals use some of the trick you are using. So to make bad guys lives more difficult, EXE and DLL's get assigned a different address every time you run a program.
If you compiled the DLL, there is an option to disable ASLR for your DLL.