I am working on some rather large C++ app, with most of the code being stored in a static library, and some programs that use that code.
I have what looks like a memory corruption run-time crash:
*** Error in `build/bin/myapp': malloc(): memory corruption (fast): 0x00000000021f62a0 ***
I want to check where that happens. GDB seems the right tool (OS: Ubuntu 14.04).
My makefiles handles both debug and release with a makefile command-line switch.
With the switch on, the -g flag is added and the .a library is 23.8 MB, while the app is 519 kB.
Without, its 1.6 MB and 486kB (so I'm pretty sure the debugging symbols are there).
My (partial) CFLAGS, as suggested by the gcc manual:
CFLAGS = -std=c++11 -g -Wall -O0 -fno-inline
I run gdb with:
gdb --args build/bin/myapp datafile.dat -a -b (...and more arguments)
My problem is that even in the debug build, gdb keeps telling me that it can't find any symbols:
Reading symbols from build/bin/myapp...(no debugging symbols found)...done.
If I run it from within gdb, it crashes with:
Program received signal SIGABRT, Aborted.
0x00007ffff5298cc9 in __GI_raise (sig=sig#entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
And the backtrace command shows indeed that symbols are missing: frames 10 to 18 have missing information, and are probably related to my code:
(gdb) bt
#0 0x00007ffff5298cc9 in __GI_raise (sig=sig#entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff529c0d8 in __GI_abort () at abort.c:89
#2 0x00007ffff52d5394 in __libc_message (do_abort=do_abort#entry=1, fmt=fmt#entry=0x7ffff53e3b28 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175
#3 0x00007ffff52e00f7 in malloc_printerr (action=<optimized out>, str=0x7ffff53e3ec8 "malloc(): memory corruption (fast)", ptr=<optimized out>) at malloc.c:4996
#4 0x00007ffff52e2e04 in _int_malloc (av=0x7ffff5620760 <main_arena>, bytes=36) at malloc.c:3359
#5 0x00007ffff52e47b0 in __GI___libc_malloc (bytes=36) at malloc.c:2891
#6 0x00007ffff5babe68 in operator new(unsigned long) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#7 0x00007ffff5c03e69 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#8 0x000000000045a7a5 in char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag) ()
#9 0x00007ffff5c05bd6 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&) ()
from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x000000000042df7f in ?? ()
#11 0x000000000042eef6 in ?? ()
#12 0x0000000000421dab in ?? ()
#13 0x0000000000422223 in ?? ()
#14 0x0000000000422cfe in ?? ()
#15 0x0000000000423393 in ?? ()
#16 0x0000000000424600 in ?? ()
#17 0x000000000040fd50 in ?? ()
#18 0x000000000040566d in ?? ()
#19 0x00007ffff5283ec5 in __libc_start_main (main=0x4053c0, argc=6, argv=0x7fffffffddf8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
stack_end=0x7fffffffdde8) at libc-start.c:287
#20 0x000000000040604f in ?? ()
I did check some of the many questions about this topic, but none of any help (most of these relate to a forgotten -g flag, or an added -s, stripping down the symbols).
Question: what can the next step be to find out why/where my crash happens?
Additional info:
gcc --version: 5.3.0
gdb --version: 7.7.1
code dependencies: boost and opencv
but none of any help (most of these relate to a forgotten -g flag, or an added -s, stripping down the symbols).
It is almost certain that you either have a stray -s somewhere on your link line, or you run stip on the binary during installation.
Look at your link command line and install command carefully, there is strip somewhere in there.
P.S. As Tom Tromey already said, GDB is rarely effective in helping with a problem like this. Using Valgrind or Address Sanitizer will likely get you to the root cause much faster.
Related
When I use perf tool to analyze my business process,It's abborted!
It's only happens to capture my business process data, other processes are normal.
I don't konw the reason.
Is there a bug with my program? how to debug?
How to use perf record normally?
Thanks for the help.
Thanks Evenyone
env:
# uname -a
Linux rk3326_64 4.4.194 #6 SMP Tue Jun 15 19:28:51 CST 2021 aarch64 GNU/Linux
# perf --version
perf version 4.4.194
perf top command and response
# perf top -p <pid>
double free or corruption (!prev)
Aborted
perf record command
# because this version perf doesn't support --sleep (specify the acquisition time ), so I use Ctrl-C to stop the perf command. when I stopped(Ctrl-C),it was aborted
# perf record -F 100 -p <pid>
double free or corruption (!prev)
Aborted
I open coredump on the device. coredump
coredump filename: core-perf-14092-6-1624288344
# gdb /usr/bin/perf ./core-perf-14092-6-1624288344
...
Core was generated by `perf top -p 12789'.
Program terminated with signal SIGABRT, Aborted.
#0 0x0000007f7f2692b8 in raise () from /lib/libc.so.6
[Current thread is 1 (LWP 14092)]
(gdb) #0 0x0000007f7f2692b8 in raise () from /lib/libc.so.6
#1 0x0000007f7f2579d4 in abort () from /lib/libc.so.6
#2 0x0000007f7f2a2040 in ?? () from /lib/libc.so.6
#3 0x0000007f7f2a862c in ?? () from /lib/libc.so.6
#4 0x0000007f7f2aa094 in ?? () from /lib/libc.so.6
#5 0x00000000004c46a4 in ?? ()
#6 0x000000000047a1b0 in ?? ()
#7 0x000000000048ab5c in ?? ()
#8 0x0000000000456520 in ?? ()
#9 0x000000000041ba68 in ?? ()
#10 0x000000000041d99c in ?? ()
#11 0x000000000044c5fc in ?? ()
#12 0x00000000004061c0 in ?? ()
#13 0x0000007f7f257e34 in __libc_start_main () from /lib/libc.so.6
#14 0x00000000004062ec in ?? ()
Backtrace stopped: not enough registers or memory available to unwind further
When I run opt with the irtranslator pass, I keep getting segfaults. Here's a variation of what I've been running:
opt -debug -mcpu=x86-64 -S sample.bc --irtranslator
Some other notes:
sample.bc is a simple hello world function I compiled into llvm bytecode with clang
some passes such as --instcombine do work
I'm using a version of llvm built from source
LLVM (http://llvm.org/):
LLVM version 11.0.0git
DEBUG build with assertions.
Default target: x86_64-unknown-linux-gnu
Host CPU: haswell
Stack Trace:
Stack dump:
0. Program arguments: ../llvm10/build/bin/opt -debug -mcpu=x86-64 -S sample.bc --x86-codegen
#0 0x00007fec32c271c7 llvm::sys::PrintStackTrace(llvm::raw_ostream&) llvm/lib/Support/Unix/Signals.inc:564:0
#1 0x00007fec32c2725a PrintStackTraceSignalHandler(void*) llvm/lib/Support/Unix/Signals.inc:625:0
#2 0x00007fec32c24ff5 llvm::sys::RunSignalHandlers() llvm/lib/Support/Signals.cpp:68:0
#3 0x00007fec32c26b44 SignalHandler(int) llvm/lib/Support/Unix/Signals.inc:406:0
#4 0x00007fec2dff2890 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12890)
#5 0x00007fec31c6e750 llvm::MachineModuleInfo::MachineModuleInfo(llvm::LLVMTargetMachine const*) llvm/lib/CodeGen/MachineModuleInfo.cpp:194:0
#6 0x00007fec31c6eddb llvm::MachineModuleInfoWrapperPass::MachineModuleInfoWrapperPass(llvm::LLVMTargetMachine const*) llvm/lib/CodeGen/MachineModuleInfo.cpp:295:0
#7 0x00007fec31c7099b llvm::Pass* llvm::callDefaultCtor<llvm::MachineModuleInfoWrapperPass>() llvm/include/llvm/PassSupport.h:80:0
#8 0x00007fec32192938 llvm::PassInfo::createPass() const llvm/include/llvm/PassInfo.h:102:0
#9 0x00007fec3218a9a2 llvm::PMTopLevelManager::schedulePass(llvm::Pass*) llvm10/llvm/lib/IR/LegacyPassManager.cpp:702:0
#10 0x00007fec3218aa07 llvm::PMTopLevelManager::schedulePass(llvm::Pass*) llvm/lib/IR/LegacyPassManager.cpp:706:0
#11 0x00007fec321933de llvm::legacy::PassManagerImpl::add(llvm::Pass*) llvm/lib/IR/LegacyPassManager.cpp:500:0
#12 0x00007fec3218f709 llvm::legacy::PassManager::add(llvm::Pass*) llvm/lib/IR/LegacyPassManager.cpp:1721:0
#13 0x00007fec2ff8650d OptCustomPassManager::add(llvm::Pass*) llvm/tools/opt/opt.cpp:340:0
#14 0x00007fec2ff7e9de addPass(llvm::legacy::PassManagerBase&, llvm::Pass*) llvm/tools/opt/opt.cpp:375:0
#15 0x00007fec2ff81013 main llvm/tools/opt/opt.cpp:862:0
#16 0x00007fec2cc51b97 __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:344:0
#17 0x00007fec2ff5150a _start (../llvm10/build/bin/opt+0x195150a)
Segmentation fault (core dumped)
IRTranslator is a pass used during the codegeneration. You're not supposed to run it directly via opt.
I tried to build webpage based on CppCMS. It compiles and works under Gentoo (GCC). But I have problems when trying to build application under FreeBSD (account on hosting provider). Default compiler is Clang.
I downloaded CppCMS package and exported few variables:
export CC=gcc48
export CXX=g++48
export LD_LIBRARY_PATH=/usr/local/lib/gcc48
Then as usual:
cmake -DCMAKE_INSTALL_PREFIX=/home/mrozigor/libs ..
make
And at this moment something is probably wrong, because it fails on all tests (make test) with segmentation fault. But I tried to build my application and used command:
PATH=$HOME/libs/bin:$PATH CPATH=$HOME/libs/include:$CPATH LIBRARY_PATH=$HOME/libs/lib:$LIBRARY_PATH ./build.sh
and it build it. But when trying to exec it (LD_LIBRARY_PATH=$HOME/libs/lib app) it also fails with SEGFAULT.
What is also interesting when i tried to execute command ldd libcppcms.so in CppCMS build directory it fails with ldd: libcppcms.so: Shared object "libcppcms.so" not found, required by "ldd", but when trying same command in lib install directory then it points to proper objects.
Is this a problem with linking some libraries? Or maybe some env variables points to some Clang dependencies etc.?
[EDIT]
Tried with options:
cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/home/mrozigor/libs -DCMAKE_C_COMPILER=gcc49 -DCMAKE_CXX_COMPILER=g++49 -DCMAKE_LIBRARY_PATH=/usr/local/lib/gcc49 ..
LD_LIBRARY_PATH=/usr/local/lib/gcc49 make
LD_LIBRARY_PATH=/usr/local/lib/gcc49
Still SEGFAULTs with given backtrace, based on dump file:
#0 0x0000000000000000 in ?? ()
#1 0x0000000802d3fc49 in __cxxabiv1::__dynamic_cast (src_ptr=0x80300f3c0 <(anonymous namespace)::ctype_c>, src_type=0x802ff35d0 <typeinfo for std::locale::facet>, dst_type=0x802ff3340 <typeinfo for std::ctype<char>>, src2dst=0) at /wrkdirs/usr/ports/lang/gcc/work/gcc-4.9.4/libstdc++-v3/libsupc++/dyncast.cc:60
#2 0x0000000802d76477 in std::has_facet<std::ctype<char> > (__loc=...) at /wrkdirs/usr/ports/lang/gcc/work/.build/x86_64-portbld-freebsd10.3/libstdc++-v3/include/bits/locale_classes.tcc:110
#3 0x0000000802d67658 in std::basic_ios<char, std::char_traits<char> >::_M_cache_locale (this=this#entry=0x606448 <std::cout##GLIBCXX_3.4+8>, __loc=...) at /wrkdirs/usr/ports/lang/gcc/work/.build/x86_64-portbld-freebsd10.3/libstdc++-v3/include/bits/basic_ios.tcc:159
#4 0x0000000802d67824 in std::basic_ios<char, std::char_traits<char> >::init (this=this#entry=0x606448 <std::cout##GLIBCXX_3.4+8>, __sb=0x80300d7c0 <__gnu_internal::buf_cout_sync>) at /wrkdirs/usr/ports/lang/gcc/work/.build/x86_64-portbld-freebsd10.3/libstdc++-v3/include/bits/basic_ios.tcc:132
#5 0x0000000802d56d51 in std::basic_ostream<char, std::char_traits<char> >::basic_ostream (__sb=<optimized out>, this=<optimized out>, __in_chrg=<optimized out>, __vtt_parm=<optimized out>) at /wrkdirs/usr/ports/lang/gcc/work/.build/x86_64-portbld-freebsd10.3/libstdc++-v3/include/ostream:85
#6 std::ios_base::Init::Init (this=<optimized out>) at /wrkdirs/usr/ports/lang/gcc/work/gcc-4.9.4/libstdc++-v3/src/c++98/ios_init.cc:91
#7 0x000000080112c941 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at /usr/local/lib/gcc49/include/c++/iostream:74
#8 0x000000080112c98c in _GLOBAL__sub_I_posix_time.cpp(void) () at /home/mrozigor/src/cppcms/booster/lib/ptime/src/posix_time.cpp:101
#9 0x000000080060ab77 in ?? () from /libexec/ld-elf.so.1
#10 0x000000080060a0ee in ?? () from /libexec/ld-elf.so.1
#11 0x0000000800608439 in ?? () from /libexec/ld-elf.so.1
#12 0x0000000000000000 in ?? ()
Code (m1.cpp):
#include <iostream>
using namespace std;
int main (int argc, char *argv[])
{
cout << "running m1" << endl;
return 0;
}
GDB Version: GNU gdb (GDB) 7.6.2
Built using: g++ -g m1.cpp
Command line history:
(gdb) b main
Breakpoint 1 at 0x40087b: file m1.cpp, line 6.
(gdb) r
Starting program: .../a.out
Program received signal SIGSEGV, Segmentation fault.
0x00002aaaaaac16a0 in strcmp () from /lib64/ld-linux-x86-64.so.2
(gdb) c
Continuing.
Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.
(gdb)
When I run without setting any breakpoints, it runs without errors.
As requested:
(gdb) bt
#0 strcmp () from /lib64/ld-linux-x86-64.so.2
#1 in check_match.12104 () from /lib64/ld-linux-x86-64.so.2
#2 in do_lookup_x () from /lib64/ld-linux-x86-64.so.2
#3 in _dl_lookup_symbol_x () from /lib64/ld-linux-x86-64.so.2
#4 in _dl_relocate_object () from /lib64/ld-linux-x86-64.so.2
#5 in dl_main () from /lib64/ld-linux-x86-64.so.2
#6 in _dl_sysdep_start () from /lib64/ld-linux-x86-64.so.2
#7 in _dl_start () from /lib64/ld-linux-x86-64.so.2
#8 in _start () from /lib64/ld-linux-x86-64.so.2
#9 in ?? ()
I was able to replicate the OP's observed behavior (using the same compile and getting the same backtrace). The behavior was persistent across a range GDBs and GCCs. I noticed that the symptom goes away when I unset SHELL. In my normal environment I use tcsh (version 1.15.00). If SHELL is set, then (I believe) gdb launches using tcsh. If I unset SHELL, gdb launches using sh. This is enough for me to make forward progress. I don't have a crisp explanation for what would be different in tcsh to manifest the issue but if others have the same behavior, it may shed more light on the issue.
I checked that in my GNU gdb version 7.11.1. It worked really fine in it.
I first compiled the same program and built it using:
g++ -g m1.cpp
Then, ran the executable in the gdb as follows:
gdb -q ./a.out
And did the same things you mentioned. It worked fine.
Update your gdb, and check that again and let know.
Pursuant to the post, Standalone functions/data in C++, I proceeded to put my "common data" in an anonymous namespace as below and everything worked great on Windows (Vista 64 bit) on VS 2005/2008/2010
namespace {
...
static std::string mystrings[] = {
str1,
str2,
...,
strN
};
...
}
namespace mynamesp {
...
use mystrings[] here..
...
}
But on Linux (so far tested RHEL5 built with GCC-4.1.2) I promptly got a segmentation fault.
$>myprog
Segmentation fault
$>gdb myprog
GNU gdb Fedora (6.8-27.el5)
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu"...
(gdb) r
Starting program: <path/to>/myprog
[Thread debugging using libthread_db enabled]
[New Thread 0x2b8901a9da60 (LWP 32710)]
Program received signal SIGSEGV, Segmentation fault.
0x0000003e4ce9c928 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string ()
from /usr/lib64/libstdc++.so.6
(gdb) bt
#0 0x0000003e4ce9c928 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string ()
from /usr/lib64/libstdc++.so.6
#1 0x00002b88ffde482b in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535)
at <path/to>/mysource.cpp:140
#2 0x00002b88ffde4d65 in global constructors keyed to _ZN91_GLOBAL__N__underscore_separated_path_to_mysource.cpp_00000000_6994A7DA2_1E () at <path/to>/mysource.cpp:12139
#3 0x00002b890011a296 in __do_global_ctors_aux ()
from <path/to/libs>/debug/libmylibd.so
#4 0x00002b88ffcd7f33 in _init () from <path/to/libs>/debug/libmylibd.so
#5 0x00002b8901672e40 in ?? ()
#6 0x000000326940d22b in call_init () from /lib64/ld-linux-x86-64.so.2
#7 0x000000326940d335 in _dl_init_internal () from /lib64/ld-linux-x86-64.so.2
#8 0x0000003269400aaa in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#9 0x0000000000000001 in ?? ()
#10 0x0000000000000000 in ?? ()
(gdb)
Line 140 in the backtrace call stack item #1 basically points to the end of my array of strings definition. I've seen some others get this error; but no obvious fixes. Appreciate any thoughts/ideas/corrections as always. Thanks!
Your problem could be releated to a static initialization order fiasco.
This happens when you initialize a static variable using another static variable. When the latter one has not been initialized yet, then the first one is using a non-initialized variable for its initialization.
The root cause is that the order, in which static variables are initialized, is undefined.
Further reading:
https://isocpp.org/wiki/faq/ctors#static-init-order
A typical workaround would be to wrap the static variables inside a function. Example:
T& GetStaticA() {
T static_var_A; // <--initialization here
return A;
}
T static_var_B = GetStaticA(); // <-- static_var_A is guaranteed to be initialized
I had this problem and it turned out that in my compiling line I had missed the final output file in the linking.
g++ main.o logger.o timer.o keyboard.o -o main -lSDL -lSDL_image -lSDL_ttf -Wall
should have been
g++ main.o logger.o timer.o keyboard.o drawer.o -o main -lSDL -lSDL_image -lSDL_ttf -Wall
(Notice the now inclusion of drawer.o?)
It was easy to miss because my actual bash compilation script had many more lines to it.