Where to look when a daemonized version of a program crashes? - c++

I have a C++ program that receives streaming music and plays it. I can run the program via the shell and run with it attached and it runs completely fine. I can stream audio to it and send things to it, but it will not crash. However, when I daemonize it and run the fork code, it will crash unexpectedly after a little bit of streaming. I try to use gdb to debug it, but it doesn't give much output.
./bin/sonar -d ; sleep 1 ; gdb ./bin/sonar $(cat sonar.pid )
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /var/sonar/bin/sonar...done.
Attaching to program: /var/sonar/bin/sonar, process 4050
Reading symbols from /usr/lib/arm-linux-gnueabihf/libcofi_rpi.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/arm-linux-gnueabihf/libcofi_rpi.so
Reading symbols from /var/sonar/bin/../lib/libboost_system.so.1.60.0...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libboost_system.so.1.60.0
Reading symbols from /var/sonar/bin/../lib/libboost_chrono.so.1.60.0...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libboost_chrono.so.1.60.0
Reading symbols from /var/sonar/bin/../lib/libboost_timer.so.1.60.0...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libboost_timer.so.1.60.0
Reading symbols from /var/sonar/bin/../lib/libboost_iostreams.so.1.60.0...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libboost_iostreams.so.1.60.0
Reading symbols from /var/sonar/bin/../lib/libboost_thread.so.1.60.0...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libboost_thread.so.1.60.0
Reading symbols from /var/sonar/bin/../lib/libboost_log_setup.so.1.60.0...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libboost_log_setup.so.1.60.0
Reading symbols from /var/sonar/bin/../lib/libboost_log.so.1.60.0...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libboost_log.so.1.60.0
Reading symbols from /var/sonar/bin/../lib/libboost_filesystem.so.1.60.0...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libboost_filesystem.so.1.60.0
Reading symbols from /var/sonar/bin/../lib/libboost_atomic.so.1.60.0...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libboost_atomic.so.1.60.0
Reading symbols from /var/sonar/bin/../lib/libboost_date_time.so.1.60.0...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libboost_date_time.so.1.60.0
Reading symbols from /var/sonar/bin/../lib/libboost_regex.so.1.60.0...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libboost_regex.so.1.60.0
Reading symbols from /var/sonar/bin/../lib/libopus.so.0...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libopus.so.0
Reading symbols from /var/sonar/bin/../lib/libmysql.so.16...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libmysql.so.16
Reading symbols from /var/sonar/bin/../lib/libmysqlpp.so.3...(no debugging symbols found)...done.
Loaded symbols for /var/sonar/bin/../lib/libmysqlpp.so.3
Reading symbols from /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.0.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.0.0
Reading symbols from /usr/lib/arm-linux-gnueabihf/libssl.so.1.0.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/arm-linux-gnueabihf/libssl.so.1.0.0
Reading symbols from /usr/lib/arm-linux-gnueabihf/libasound.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/arm-linux-gnueabihf/libasound.so.2
Reading symbols from /usr/lib/arm-linux-gnueabihf/libstdc++.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/arm-linux-gnueabihf/libstdc++.so.6
Reading symbols from /lib/arm-linux-gnueabihf/libm.so.6...Reading symbols from /usr/lib/debug/lib/arm-linux-gnueabihf/libm-2.19.so...done.
done.
Loaded symbols for /lib/arm-linux-gnueabihf/libm.so.6
Reading symbols from /lib/arm-linux-gnueabihf/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/arm-linux-gnueabihf/libgcc_s.so.1
Reading symbols from /lib/arm-linux-gnueabihf/libpthread.so.0...Reading symbols from /usr/lib/debug/lib/arm-linux-gnueabihf/libpthread-2.19.so...done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
[New Thread 0x749ff450 (LWP 4054)]
[New Thread 0x75358450 (LWP 4053)]
[New Thread 0x75b58450 (LWP 4052)]
done.
Loaded symbols for /lib/arm-linux-gnueabihf/libpthread.so.0
Reading symbols from /lib/arm-linux-gnueabihf/libc.so.6...Reading symbols from /usr/lib/debug/lib/arm-linux-gnueabihf/libc-2.19.so...done.
done.
Loaded symbols for /lib/arm-linux-gnueabihf/libc.so.6
Reading symbols from /lib/arm-linux-gnueabihf/librt.so.1...Reading symbols from /usr/lib/debug/lib/arm-linux-gnueabihf/librt-2.19.so...done.
done.
Loaded symbols for /lib/arm-linux-gnueabihf/librt.so.1
Reading symbols from /lib/arm-linux-gnueabihf/libbz2.so.1.0...(no debugging symbols found)...done.
Loaded symbols for /lib/arm-linux-gnueabihf/libbz2.so.1.0
Reading symbols from /lib/arm-linux-gnueabihf/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/arm-linux-gnueabihf/libz.so.1
Reading symbols from /lib/arm-linux-gnueabihf/libdl.so.2...Reading symbols from /usr/lib/debug/lib/arm-linux-gnueabihf/libdl-2.19.so...done.
done.
Loaded symbols for /lib/arm-linux-gnueabihf/libdl.so.2
Reading symbols from /lib/ld-linux-armhf.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux-armhf.so.3
Reading symbols from /lib/arm-linux-gnueabihf/libnss_files.so.2...Reading symbols from /usr/lib/debug/lib/arm-linux-gnueabihf/libnss_files-2.19.so...done.
done.
Loaded symbols for /lib/arm-linux-gnueabihf/libnss_files.so.2
0x763e1d14 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
81 ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) c
Continuing.
Program received signal SIGABRT, Aborted.
[Switching to Thread 0x75358450 (LWP 4053)]
0x7633df70 in raise () from /lib/arm-linux-gnueabihf/libc.so.6
(gdb) bt
#0 0x7633df70 in raise () from /lib/arm-linux-gnueabihf/libc.so.6
#1 0x7633f324 in abort () from /lib/arm-linux-gnueabihf/libc.so.6
#2 0x7656eb5c in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/arm-linux-gnueabihf/libstdc++.so.6
#3 0x7656c9a0 in ?? () from /usr/lib/arm-linux-gnueabihf/libstdc++.so.6
#4 0x7656c9a0 in ?? () from /usr/lib/arm-linux-gnueabihf/libstdc++.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) c
Continuing.
[Thread 0x762ac000 (LWP 4050) exited]
[Thread 0x75b58450 (LWP 4052) exited]
[Thread 0x749ff450 (LWP 4054) exited]
Program terminated with signal SIGABRT, Aborted.
The program no longer exists.
(gdb)
The daemonizing code that I use:
// Daemonize
LOG_INFO( *logger ) << "Daemonizing.";
// Sig stuff
pid_t pid, sid;
// Start the forky pig
if( (pid = fork())<0 )
{
LOG_ERROR( *logger ) << "Error forking: " << strerror( errno );
return( EXIT_FAILURE );
}
// The parent can go home
if( pid>0 )
return( EXIT_SUCCESS );
// Make ourselves emancipate
if( (sid = setsid())<0 )
{
LOG_ERROR( *logger ) << "setsid error: " << strerror( errno );
return( EXIT_FAILURE );
}
// Ignore sigpipes
signal( SIGPIPE, SIG_IGN );
// Close the file descriptors to detach
close( STDIN_FILENO );
close( STDOUT_FILENO );
close( STDERR_FILENO );
If the program works when it's not daemonized and crashes when it is, there is obviously some difference in the way that it runs. So my question really is, where can I look for this particular point of failure?
Here is the same error on a 64-bit intel arch processor:
Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7f80777b8700 (LWP 10314)]
0x00007f807baa7297 in __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
55 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007f807baa7297 in __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
#1 0x00007f807baa862a in __GI_abort () at abort.c:89
#2 0x00007f807c5a700d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/libstdc++.so.6
#3 0x00007f807c5a4e96 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/libstdc++.so.6
#4 0x00007f807c5a4ee1 in std::terminate() () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/libstdc++.so.6
#5 0x00007f807c600c21 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/libstdc++.so.6
#6 0x00007f807be16324 in start_thread (arg=0x7f80777b8700) at pthread_create.c:333
#7 0x00007f807bb5bf6d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

there is a difference in running a process in an interactive environment or as a daemon. When you run a process from the command line, then the process is inheriting all resources from the interactive environment like Paths, defined in environment PATH variables, or even the working directory, which is, in an interactive environment, the actual directory from where a process is started.
Without knowing your system or environment, one of the mistakes which are often done is trying to start background process, daemons, for example from a crontab, but to forget to give the information a process needs to work. When a process is running as a daemon, without knowing the circumstances of your work flow, for example, it i necessary to give the process the environment, to mention one, the working directory. I do it for example with "chdir("/");", so the forked child will know, on what working directory he is active.
Depending on your other resources, you should give the daemon all the information, it needs to fulfil its job, e.g. if it starts some process, for example the PATH to the process and so on.
regards

Turns out that it was a log file crash. I wasn't seeing the throw error because the error was with the logging system itself. Since all of the ties to stdout were cut, nothing was being displayed. It was exactly the issue in this post with Boost.Log:
Boost Log causes crash when trying first log statement (when not Administrator)

Related

What is This Error I get everytime after Debugging everytime and It's g++ compiler I'm using

The Error I Get After Debugging, When I press a key random key to close the terminal in VS code (integrated) It automatically starts Powershell (integrated) and shows this:
PS C:\Users\####\Desktop\CPP> & 'c: \Users\###\.vscode\extensions\ms-vscode.cpptools-1.5.1\debugAdapters\bin\WindowsDebugLauncher.exe' '--stdin=Microsoft-MIEngine-nukgex502.ouf' '--stdout=Microsoft-MIEngine-Out-cdrwtdkv.2c4' '--stderr=Microsoft-MIEngine-Error-yiwuqa2b.xmu' '--pid=licrosoft-MIEngine-Pid-iyaavmoy.sbg' '--dbgExe=C: \Program Files\minguW-W64 \mingw64 \bin \gdb.exe ''--interpreter=mi' -2147483643 2147493647
This All thing in Debug Consol, After Powershell:
=thread-group-added,id="i1"
GNU gdb (GDB) 8.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-w64-mingw32".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Warning: Debuggee TargetArchitecture not detected, assuming x86_64.
=cmd-param-changed,param="pagination",value="off"
[New Thread 10432.0x3cd8]
[New Thread 10432.0xe90]
[New Thread 10432.0x24a8]
[New Thread 10432.0x3bc0]
Thread 1 hit Breakpoint 1, main () at C:\Users\#####\Desktop\Cpp\Binary_conversion.cpp:18
18 return 0;
Loaded 'C:\WINDOWS\SYSTEM32\ntdll.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\System32\kernel32.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\System32\KernelBase.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\SYSTEM32\apphelp.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\System32\msvcrt.dll'. Symbols loaded.
Loaded 'C:\Program Files\mingw-w64\mingw64\bin\libstdc++-6.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\System32\user32.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\System32\win32u.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\System32\gdi32.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\System32\gdi32full.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\System32\msvcp_win.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\System32\ucrtbase.dll'. Symbols loaded.
Loaded 'C:\Program Files\mingw-w64\mingw64\opt\bin\libgcc_s_seh-1.dll'. Symbols loaded.
Loaded 'C:\Program Files\mingw-w64\mingw64\opt\bin\libwinpthread-1.dll'. Symbols loaded.
Loaded 'C:\WINDOWS\System32\imm32.dll'. Symbols loaded.
[Thread 10432.0x24a8 exited with code 0]
[Thread 10432.0x3bc0 exited with code 0]
[Thread 10432.0xe90 exited with code 0]
[Inferior 1 (process 10432) exited normally]
The program 'c:\Users\#####\Desktop\Cpp\Binary_conversion.exe' has exited with code 0 (0x00000000).
IDK what is this, So can anyone explain to me what is this and How to fix this?

GDB reading symbols with "symbol-file" command on a core file

I am trying to analyze segfault on a core file on linux. I am not sure if the following behavior is correct, thus i deliberately caused a segfault using
#include <signal.h>
int main() {
raise(SIGSEGV);
}
the binary is build with debug info i.e.
file mainTestFile
mainTestFile: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/l, for GNU/Linux 3.2.0, with debug_info, not stripped
notice how it does say "with debug_info, not stripped" at the end
when i execute the binary i get a core file generated which is called core-mainTestFile.20474
(In order to generate the core file i hat to set my ulimit to unlimited i.e.
ulimit -c unlimited
)
if i run only the binary under GDB and do backtrace "bt" then i get the segfault and i get all names of the functions involved
printed nicely i.e. notice how the gdb says when starting "reading symbols from ./mainTestFile...done."
gdb ./mainTestFile
GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git
....
reading symbols from ./mainTestFile...done.
(gdb) run
Starting program: /src/exe/mainTestFile
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Program received signal SIGSEGV, Segmentation fault.
__GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 __GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x0000000000402dad in main (argc=1, argv=0x7fffffffda38) at /src/exe/main.cpp:53
(gdb)
however if i try to anaylise only the core file with gdb like that
gdb -c core-mainTestFile.20474
then i get only question marks
the when i execute "bt" then i do not see the names of the methods, instead i get question marks
(gdb) bt
#0 0x00007f34d8842e97 in ?? ()
#1 0x0000000000000000 in ?? ()
they only workaround i found is to supply the binary directly at the command line then it all gets printed nicely.
even if i try to tell GDB to use the symbols file and point that to the binary file which does have the symbols
i.e.
symbol-file /src/exe/mainTestFile
then GDB says
Reading symbols from /src/exe/mainTestFile...done
and when i execute bt i see the question marks again? Why is that. Is GDB not able to get the symbols out of the binary?
it only works if supply the binary directly on the command like like:
gdb /src/exe/mainTestFile -c core-mainTestFile.20474
my question is should the GDB be able to read symbols of the binary when directly supplying him the binary over the "symbol-file" command or not. Why is this working when supping him directly the binary over the command line, what is the difference?
should the GDB be able to read symbols of the binary when directly supplying him the binary over the "symbol-file" command or not.
In theory, using symbol-file and core-file commands in either order in GDB should be equivalent.
But there is a bug: symbol-file followed by core-file works, and the opposite order doesn't.
Since generally the end-user can always rearrange his commands into the order that works, this has never propagated to the top of any GDB developer's queue of things to fix.
Related bug (but not an exact duplicate).

GDB with corefile on remote embedded device - How to get more information about backtrace?

I have a core dump from a C++ application running on an embedded imx6 board (yocto linux). I can put gdb on the box and run it in a terminal to examine the core file like so just fine:
gdb myApplication core.udpsrc256:src.1520419431.5526
I get extremely limited information, and really need to know more about what caused the core dump. All I have is a printout from the application:
(myApplication:5526): GLib-ERROR **: ../../glib-2.46.2/glib/gmem.c:100: failed to allocate 65611 bytes
./run-app.sh: line 8: 5526 Trace/breakpoint trap (core dumped) XDG_RUNTIME_DIR=/run/user/root ./myApplication
Also the core dump backtrace gives some useless stuff. I need to know more stuff up the stack that led to this frame:
#0 0x75ff1910 in raise () from /lib/libc.so.6
[Current thread is 1 (LWP 5533)]
(gdb)
(gdb)
(gdb) bt
#0 0x75ff1910 in raise () from /lib/libc.so.6
#1 0x6b169558 in g_logv () from /usr/lib/libglib-2.0.so.0
#2 0x6b169610 in g_log () from /usr/lib/libglib-2.0.so.0
#3 0x6b1681c4 in g_malloc () from /usr/lib/libglib-2.0.so.0
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
Sidenote -- there is some warnings when I startup gdb:
GNU gdb (GDB) 7.10.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-poky-linux-gnueabi".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from qt5qmlvideo...done.
warning: exec file is newer than core file.
[New LWP 5533]
[New LWP 5526]
[New LWP 5531]
[New LWP 5528]
[New LWP 5534]
[New LWP 21064]
[New LWP 5536]
[New LWP 21065]
[New LWP 5532]
[New LWP 5527]
[New LWP 5530]
[New LWP 5537]
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
Core was generated by `./qt5qmlvideo -platform wayland'.
Program terminated with signal SIGTRAP, Trace/breakpoint trap.
#0 0x75ff1910 in raise () from /lib/libc.so.6
[Current thread is 1 (LWP 5533)]
(gdb)
Can anyone help? Do I need some of the stuff gdb warns about... or can i rebuild my application and its dependencies in some other configuration that would give more output? Thank you!
Some more notes that may matter -
This is a multithreaded application running a gstreamer pipeline. Many gstreamer plugins generate their own threads, one of which in this pipeline is 'udpsrc'. I'm wondering if it's because this failure happens in one of those threads is the reason why I can't get details, but I want to know how to get it to show the details if possible!
(1)
The
Do you need "set solib-search-path" or "set sysroot"?
is a problem. Check the path (on your device) where linux-vdso.so.1 resides, and include that in the solib-search-path. Similarly for the other shared-object libraries that your program uses. E.g. if some shared-object libraries are in /lib, some are in /usr/adowdy/lib and some are in /usr/adowdy/arm/lib, you can say:
(gdb) set solib-search-path /lib:/usr/adowdy/lib:/usr/adowdy/arm/lib
(2) The
warning: Unable to find libthread_db matching inferior's thread
library, thread debugging will not be available.
is also a problem. See the answer to this question
(3) The
failed to allocate 65611 bytes
is a clue. Are you, by any chance, trying to allocate a negative number of bytes (maybe 65536 - 65611 = -75 bytes)?
Also the core dump backtrace gives some useless stuff.
It's not entirely useless. The stack trace, and the message from the application say the same thing: your application ran out of memory (malloc failed to allocate 65611 bytes).
While a more complete stack would tell you which particular call to g_malloc failed, it's very likely to not matter in practice -- if this g_malloc didn't fail, the next one would.
You likely have a memory leak, or are simply allocating too much memory for what your system allows.
You should look into many debugging tools built for solving this exact problem.

specify glibc on debugging core dump with gdb

I have a program that crashed on RHEL 6.5, also I have a coredump file.
I want to debug it on RHEL 6.4, but it has another version of glibc. I put glibc.so.6 from RHEL 6.5 to lib/ directory, but can't set gdb up to use it:
[nskdvlp#srv2-csd-mong01 snmp_int]$ LD_LIBRARY_PATH=/u01/app/maa/snmp_int/lib gdb bin/snmp_int
Reading symbols from /u01/app/maa/snmp_int/bin/snmp_int...done.
(gdb) info sharedlibrary
No shared libraries loaded at this time.
(gdb) set solib-search-path lib
warning: .dynamic section for "/lib64/libc.so.6" is not at the expected address (wrong library or version mismatch?)
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /u01/app/maa/snmp_int/lib/libsnmp_int_snmp.so.6.0.5...done.
Loaded symbols for /u01/app/maa/snmp_int/lib/libsnmp_int_snmp.so.6.0.5
(gdb) core core.110358
[New Thread 110358]
[New Thread 110367]
warning: .dynamic section for "/lib64/libc.so.6" is not at the expected address (wrong library or version mismatch?)
warning: Could not load shared library symbols for (0x%0*Zx)
.
Do you need "set solib-search-path" or "set sysroot"?
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
(gdb) bt
#0 0x0000003e93a32925 in ?? ()
#1 0x0000003e93a3408d in ?? ()
#2 0x0000000000000000 in ?? ()
Other .so libraries are loaded well from lib/
I put glibc.so.6 from RHEL 6.5 to lib/ directory, but can't set gdb up to use it:
There is no such thing as glibc.so.6. You mean libc.so.6.
This is nowhere near sufficient: you need all libraries and the dynamic loader from the system on which the core was produced. See this answer for what you need to do.

Debugging Firebreath using gdb in Linux

The issue is simple. I have my plugin created by Firebreath and would like to debug my *API.cpp file using gdb in Linux.
Yes, i already read this article, and i got this PID:
30227 0.0 0.4 115024 12300 pts/2 Sl+ 00:03 0:00 /usr/lib/chromium/chromium --type=plugin --plugin-path=/home/USER/Documents/mycode/0.1/npMyPlugin.so
When i opened with gdb i always get a similar output like this:
Reading symbols from /usr/lib/libpcrecpp.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libpcrecpp.so.0
Reading symbols from /usr/lib/libssh2.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libssh2.so.1
Reading symbols from /usr/lib/libssl.so.1.0.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libssl.so.1.0.0
Reading symbols from /usr/lib/libcrypto.so.1.0.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libcrypto.so.1.0.0
Probably not the right process. Any idea?
Thanks in advanced.
Update - Ignoring the message
Thanks guys for your comments. Well i ignored the message and did the following:
(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n)
Program not restarted.
(gdb) b 328
No line 328 in the current file.
Make breakpoint pending on future shared library load? (y or n)
So basically gdb load my MyPlugin.cpp and not my MyPluginAPI.cpp. Any idea?
Setting a break point with just a line-number only works if you have exactly one source file.
Use e.g. b MyPluginAPI.cpp:328 or b MyPluginAPI::myMethod() instead.