Here is the problem which I'm trying to address:
We've got a core dump while processing data. The result of backtracing is:
#0 0x00a99402 in __kernel_vsyscall ()
#1 0x00306df0 in raise () from /lib/libc.so.6
#2 0x00308701 in abort () from /lib/libc.so.6
#3 0x001c4530 in _gnu_cxx::_verbose_terminate_handler() () from /usr/lib/libstdc++.so.6
#4 0x001c1f35 in ?? () from /usr/lib/libstdc++.so.6
#5 0x001c12ca in ?? () from /usr/lib/libstdc++.so.6
#6 0x001c1d99 in __gxx_personality_v0 () from /usr/lib/libstdc++.so.6
#7 0x00d1c7e6 in ?? () from /lib/libgcc_s.so.1
#8 0x00d1cb62 in _Unwind_Resume () from /lib/libgcc_s.so.1
........
I've looked through the code base of our application and it is not clear that the problem is due to uncaught exception or whatever (but i know it somehow connected with exceptions because of _Unwind_Resume call is there). So I'm trying to write simple program which also fails with core dump and its gdb backtracing contains the lines above.
os: CentOS, compiler: gnu gcc 4.1.2, language: c/c++
Any suggestions about the problem/code would be much appreciated
Related
I am trying to debug my c++ programming assignment application using gdb on an Ubuntu server, because it produces segmentation fault.
But the file produces ?? symbols that are unreadable to me when I try bt it gives me.
(gdb) bt
#0 0x00007f141956d277 in ?? ()
#1 0x00007ffc1a866bd0 in ?? ()
#2 0x000055e1f101d5e0 in ?? ()
#3 0x00007ffc1a866db0 in ?? ()
#4 0x000055e1f1433e70 in ?? ()
#5 0x00007ffc1a866bd0 in ?? ()
#6 0x000055e1f10224a9 in ?? ()
#7 0x000055e1f14341f8 in ?? ()
#8 0x00000001f14344d0 in ?? ()
#9 0x0000000000000000 in ?? ()
I was following this link, and it told me to load these symbols
symbol-file /path/to/my/binary
sharedlibrary
The sharedlibrary was found, but the symbol-file path is not there. So,it did change bt command output somehow
(gdb) bt
#0 tcache_get (tc_idx=0) at malloc.c:2943
#1 _GI__libc_malloc (bytes=19) at malloc.c:3050
#2 0x000055e1f10224a9 in ?? ()
#3 0x000055e1f14341f8 in ?? ()
#4 0x00000001f14344d0 in ?? ()
#5 0x0000000000000000 in ?? ()
I still don't understand the bug.
Now, I don't know it's a problem from the GDB for not having this symbol-file or its a compilation problem which I don't know how or that's enough for me to debug, but I was following Debugging a Segmentation Fault and it was much clearer to troubleshoot.
When I search for similar cases, all of them were answered only for their case, not a general solution how to deal with these kinds of error. I also thought of installing or locating that symbol-file but I didn't understand how.
If someone could help me, I need to understand what is my problem and how should I fix it.
Note: core dump is produced in the /tmp not in current application directory
I was following this link, and it told me to load these symbols
Don't follow this link (it's unnecessarily complicated for your use case).
Instead, do this:
gdb /path/to/my/binary
(gdb) run
... GDB will stop when your program encounters SIGSEGV
(gdb) bt # should produce meaningful output now.
Recently I upgraded my OS from Ubuntu Precise Saucy (13.10) to Trusty (14.04). After this upgrade, cv::VideoCapture became not working properly. The program aborts when reading a video file. For example,
int main(int argc, char**argv)
{
cv::VideoCapture vin("sample/vout2l.avi");
...
Executing this program, it aborts with a message:
*** Error in `./cv2-videoread.out': malloc(): memory corruption: 0x0000000000e3eff0 ***
Abort (core dumped)
The backtrace looks like:
[New LWP 15586]
[New LWP 15587]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `./cv2-videoread.out'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007ff953e61c37 in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0 0x00007ff953e61c37 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007ff953e65028 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007ff953e9e2a4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007ff953eabe26 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x00007ff953eac1ab in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#5 0x00007ff953eadba4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#6 0x00007ff953eaf7d2 in posix_memalign () from /lib/x86_64-linux-gnu/libc.so.6
#7 0x00007ff94fa640fe in av_malloc () from /usr/lib/x86_64-linux-gnu/libavutil.so.52
#8 0x00007ff94fa641b1 in av_strdup () from /usr/lib/x86_64-linux-gnu/libavutil.so.52
#9 0x00007ff94fa5e5db in av_dict_set ()
from /usr/lib/x86_64-linux-gnu/libavutil.so.52
#10 0x00007ff954738574 in CvCapture_FFMPEG::open(char const*) ()
from /usr/lib/libopencv_highgui.so.2.4
#11 0x00007ff954738719 in cvCreateFileCapture_FFMPEG ()
from /usr/lib/libopencv_highgui.so.2.4
#12 0x00007ff95473aac9 in cvCreateFileCapture_FFMPEG_proxy(char const*) ()
from /usr/lib/libopencv_highgui.so.2.4
---Type <return> to continue, or q <return> to quit---
#13 0x00007ff954722d89 in cvCreateFileCapture ()
from /usr/lib/libopencv_highgui.so.2.4
#14 0x00007ff954723045 in cv::VideoCapture::open(std::string const&) ()
from /usr/lib/libopencv_highgui.so.2.4
#15 0x00007ff95472315c in cv::VideoCapture::VideoCapture(std::string const&) ()
from /usr/lib/libopencv_highgui.so.2.4
#16 0x0000000000401281 in main (argc=1, argv=0x7fff1f938388) at cv2-videoread.cpp:30
(gdb)
NOTE: cv::VideoCapture vin(... is 30th line.
Before upgrading the OS, this code was working with the same input file.
From the backtrace, it seems that the trouble happens at CvCapture_FFMPEG and libavutil. I upgraded the packages ffmpeg libavutil-dev libavutil51 libavutil52 but they were already up-to-date.
Also, OpenCV packages are up-to-date (I checked libopencv-core-dev libopencv-core2.4 libopencv-dev libopencv-highgui-dev libopencv-highgui2.4).
I also tested OpenCV built from source, but got the same results.
Do you have ideas to figure this out?
So, I've solved this issue.
By analyzing the program with ldd, I found that it was linked to, for example, /usr/lib/libopencv_highgui.so. However, in x86_64 system, it should be /usr/lib/x86_64-linux-gnu/libopencv_highgui.so. In my system, both files were installed.
The issue was caused by /usr/lib/libopencv_*.so (I'm not sure how I installed them. Maybe from source code...?). I removed these files, and compiled the above program again. Then it worked without errors.
C++ application gets crashed with the core file showing error
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff79e54000
Core was generated by `./server'.
Program terminated with signal 6, Aborted.
#0 0x0000003b67230265 in raise () from /lib64/libc.so.6
(gdb) bt
#0 0x0000003b67230265 in raise () from /lib64/libc.so.6
#1 0x0000003b67231d10 in abort () from /lib64/libc.so.6
#2 0x0000003b6726a9bb in __libc_message () from /lib64/libc.so.6
#3 0x0000003b6727247f in _int_free () from /lib64/libc.so.6
#4 0x0000003b672728db in free () from /lib64/libc.so.6
#5 0x00000000004060df in operator delete (p=0x20030190) at ../lib/m_string.cpp:43
#6 0x0000000000403892 in TStr::~TStr (this=0x2102c980, __in_chrg=<value optimized out>) at ../lib/m_string.cpp:175 –
could able to understand about this issue. Here is the link that i have verified https://bugzilla.redhat.com/show_bug.cgi?id=959013
shows that the size of vdso file is not enough. which is in path /proc/self/maps.
please let me know what kind of issue is this and please suggest a fix for this.
what kind of issue is this
Any crash inside malloc or free is a sure sign of previous heap corruption.
Use Valgrind or AddressSanitizer (incorporated into GCC-4.8 as well) to find the root cause.
Ignore vdso -- as Tom Tromey said, it has nothing to do with the problem.
WARNING The code I provide in my question may crash matlab AND your machine!!!
I have written a mex gateway function to a C++ class. If I compile this mex function on 32Bit windows using R2008a I have no problems. If I compile and run on Matlab R2011a running 64bit Scientific Linux (a version of Red Hat Enterprize Linux) matlab exits with a segfault when the mexfunction is called, although it appears to run about halfway through the program. The C++ class can be compiled and run (with a main function) outside of Matlab on both platforms with no errors. I am using Microsoft Visual C++ Express Edition on the windows machine, and gcc 4.4.5 on the Linux machine.
What is the cause of this and can I fix it?
A zip file containing the code and data files necessary to reproduce the problem can be downloaded from http://www.see.ed.ac.uk/~s0237326/downloads/mexcrash.zip. This zip file contains the .m and .cpp source code, and a text file for testing (Temp.fem, 10kB). The file fmeshersetup.m shows the commands I am using to compile. The file Test_mexfmesher.m runs the mexfunction with an appropriate input for testing. The mex gateway function is mexfmesher.cpp, it calls the fmesher class which is made up of the files in the fmesher directory.
There are a total of 13 C++ source files, which is a bit much to expect someone to debug, even the particular line on which the segfault occurs woud be of interest.
Unfortunately I do not have access to a graphical debugger on 64bit linux that can interface with Matlab, and was hoping someone would immediately see the problem, or it would be a known problem. As stated, when run not as a mex file, the program runs fine, so I cannot locate the error in another tool, and I'm not very familiar with gdb. I believe the problem might be related to a calls to the C++ function 'free'.
Below is a backtrace from the segfault:
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff721be36 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwservices.so
(gdb) backtrace
#0 0x00007ffff721be36 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwservices.so
#1 0x00000034daa914f2 in std::basic_ostream<char, std::char_traits<char> >::flush() () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libstdc++.so.6
#2 0x00007ffff72302b0 in ioFlush () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwservices.so
#3 0x00007ffff61303f5 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#4 0x00007ffff61330ec in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#5 0x00007ffff6130c7a in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#6 0x00007ffff6131741 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#7 0x00007ffff618a7d9 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#8 0x00007ffff686d7ef in Mfh_file::dispatch_fh(int, mxArray_tag**, int, mxArray_tag**) () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_dispatcher.so
#9 0x00007ffff61731f0 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#10 0x00007ffff6114975 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#11 0x00007ffff612e96e in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#12 0x00007ffff61330ec in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#13 0x00007ffff6130c7a in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#14 0x00007ffff6131741 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#15 0x00007ffff618a7d9 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#16 0x00007ffff686d7ef in Mfh_file::dispatch_fh(int, mxArray_tag**, int, mxArray_tag**) () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_dispatcher.so
#17 0x00007ffff61669b2 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#18 0x00007ffff6128e13 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#19 0x00007ffff6127eb7 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#20 0x00007ffff6128397 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwm_interpreter.so
#21 0x00007ffff6d378fe in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwbridge.so
#22 0x00007ffff6d384ae in mnParser () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwbridge.so
#23 0x00007ffff6ae0d39 in mcrInstance::mnParser_on_interpreter_thread() () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwmcr.so
#24 0x00007ffff6ac3db2 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwmcr.so
#25 0x00007ffff6ac3ec0 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwmcr.so
#26 0x00007fffee8badb6 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/../../bin/glnxa64/libmwuix.so
#27 0x00007fffee8c413d in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/../../bin/glnxa64/libmwuix.so
#28 0x00007fffef3110bd in sysq::during_F<boost::weak_ptr<sysq::ws_ppeHook>, boost::shared_ptr<sysq::ws_ppeHook> > std::for_each<__gnu_cxx::__normal_iterator<boost::weak_ptr<sysq::ws_ppeHook>*, std::vector<boost::weak_ptr<sysq::ws_ppeHook>, std::allocator<boost::weak_ptr<sysq::ws_ppeHook> > > >, sysq::during_F<boost::weak_ptr<sysq::ws_ppeHook>, boost::shared_ptr<sysq::ws_ppeHook> > >(__gnu_cxx::__normal_iterator<boost::weak_ptr<sysq::ws_ppeHook>*, std::vector<boost::weak_ptr<sysq::ws_ppeHook>, std::allocator<boost::weak_ptr<sysq::ws_ppeHook> > > >, __gnu_cxx::__normal_iterator<boost::weak_ptr<sysq::ws_ppeHook>*, std::vector<boost::weak_ptr<sysq::ws_ppeHook>, std::allocator<boost::weak_ptr<sysq::ws_ppeHook> > > >, sysq::during_F<boost::weak_ptr<sysq::ws_ppeHook>, boost::shared_ptr<sysq::ws_ppeHook> >) ()
from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/../../bin/glnxa64/libuij.so
#29 0x00007fffef312989 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/../../bin/glnxa64/libuij.so
#30 0x00007fffef30f4ae in svWS_ProcessPendingEvents(int, int, bool) () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/../../bin/glnxa64/libuij.so
#31 0x00007ffff6ac21c7 in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwmcr.so
#32 0x00007ffff6ac260a in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwmcr.so
#33 0x00007ffff6ac2d6f in ?? () from /opt/matlab-2011a/bin/glnxa64/../../bin/glnxa64/libmwmcr.so
#34 0x00000034d7a077e1 in start_thread () from /lib64/libpthread.so.0
#35 0x00000034d72e573d in clone () from /lib64/libc.so.6
(gdb) quit
You have attempted to use 'free' to free memory with a pointer which actually points to NULL. A careful programmer would check if this is the case before attempting to use free. Some libraries will have an automatic check for this.
The reason you did not notice this when compiling as a standalone may be due to the fact that matlab uses its own libstdc++ (stored in /opt/matlab-2011a/bin/glnxa64/libstdc++.so.6) instead of the main one (stored in /usr/lib/x86_64-linux-gnu/).
If you are sudoer, You should force matlab to use the main libstdc++, this can solve many problems with mex files:
cd /opt/matlab-2011a/bin/glnxa64/
sudo mkdir old
sudo mv libstdc++.so.6* old
sudo ln -s /usr/lib/x86_64-linux-gnu/libstdc++.so.6 libstdc++.so.6
I'm playing with X-development. I've got a basic proto-WM going that works for a while and then produces these errors after a fairly random time.
[edit]
Locking assertion failure. Backtrace:
#0 /usr/lib/libxcb-xlib.so.0 [0x7f71dcf9a9ac]
#1 /usr/lib/libxcb-xlib.so.0(xcb_xlib_unlock+0x24) [0x7f71dcf9aa54]
#2 /usr/lib/libX11.so.6 [0x7f71ddefe340]
#3 /usr/lib/libX11.so.6(XAllocColor+0xc1) [0x7f71ddedada1]
#4 /home/mais/code/simplewin/bin/Debug/simplewin [0x408e0b]
#5 /home/mais/code/simplewin/bin/Debug/simplewin [0x409062]
#6 /home/mais/code/simplewin/bin/Debug/simplewin [0x407a9d]
#7 /home/mais/code/simplewin/bin/Debug/simplewin [0x406c6d]
#8 /home/mais/code/simplewin/bin/Debug/simplewin [0x402734]
#9 /home/mais/code/simplewin/bin/Debug/simplewin [0x407e37]
#10 /home/mais/code/simplewin/bin/Debug/simplewin [0x407304]
#11 /home/mais/code/simplewin/bin/Debug/simplewin [0x407335]
#12 /lib/libpthread.so.0 [0x7f71ddc9afc7]
#13 /lib/libc.so.6(clone+0x6d) [0x7f71dd26a59d]
Locking assertion failure. Backtrace:
#0 /usr/lib/libxcb-xlib.so.0 [0x7f71dcf9a9ac]
#1 /usr/lib/libxcb-xlib.so.0(xcb_xlib_lock+0x17) [0x7f71dcf9ab17]
#2 /usr/lib/libX11.so.6 [0x7f71ddefe420]
#3 /usr/lib/libX11.so.6 [0x7f71ddefeb5b]
#4 /usr/lib/libX11.so.6 [0x7f71ddefeeb5]
#5 /usr/lib/libX11.so.6(XNextEvent+0x68) [0x7f71ddee5898]
#6 /home/mais/code/simplewin/bin/Debug/simplewin [0x404bc2]
#7 /lib/libpthread.so.0 [0x7f71ddc9afc7]
#8 /lib/libc.so.6(clone+0x6d) [0x7f71dd26a59d]
It looks like some kind of synchronization bug in my app; that much of the time the resources used are released in the correct sequence, but at some stage this get mixed-up and then the error(s) occur.
How do I interpret the above to find the location/cause of the errors?
1) build your program with debug symbold (on gcc, pass -g option)
2) enable core files, and once your application crashes, open it in gdb with core file, and check the backtrace
3) is your application multithreaded and using xlib? If so, you need to enable multi-threading access to the xlib functions.
4) install ddd, and run your application using that (ddd is a gui frontend for the linux debuger gdb)