coredump at __correctly_grouped_prefixwc - c++

the program under live env, segmentation fault some time, i try to gdb the coredump file,
but can't found the code line cause coredump.
Program terminated with signal 11, Segmentation fault.
#0 0x00000038f3a41bf5 in __correctly_grouped_prefixwc () from /lib64/libc.so.6
(gdb) bt
#0 0x00000038f3a41bf5 in __correctly_grouped_prefixwc () from /lib64/libc.so.6
#1 0x0000000000000000 in ?? ()
(gdb) info r
rax 0x1ac1b108 448901384
rbx 0x2add423b4ff0 47129787322352
rcx 0x2add48128640 47129885312576
rdx 0x0 0
rsi 0x1 1
rdi 0x2add48000020 47129884098592
rbp 0x2add3f1aef50 0x2add3f1aef50
rsp 0x2add423b4ff0 0x2add423b4ff0
r8 0x2 2
r9 0x2 2
r10 0x0 0
r11 0x0 0
r12 0x0 0
r13 0x3 3
r14 0x1000 4096
r15 0x2add3f1b0000 47129734873088
rip 0x38f3a41bf5 0x38f3a41bf5 <__correctly_grouped_prefixwc+165>
eflags 0x10246 [ PF ZF IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
fctrl 0x37f 895
fstat 0x0 0
ftag 0xffff 65535
fiseg 0x0 0
fioff 0xc54f06 12930822
foseg 0x2add 10973
fooff 0x423b3f00 1111179008
fop 0x0 0
mxcsr 0x1fa1 [ IE PE IM DM ZM OM UM PM ]
cat /etc/redhat-release
CentOS release 5.5 (Final)
and i want to debug glibc at the source level, run yum install yum-utils to install the debuginfo-install program.
then, run sudo debuginfo-install glibc, the result following
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* addons: centos.ustc.edu.cn
* base: mirror.bit.edu.cn
* extras: centos.ustc.edu.cn
* updates: centos.ustc.edu.cn
Checking for new repos for mirrors
Could not find debuginfo for main pkg: glibc-2.5-123.x86_64
Could not find debuginfo for main pkg: glibc-2.5-123.i686
No debuginfo packages available to install
and then i try to run yum search glibc-debuginfo
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* addons: centos.ustc.edu.cn
* base: mirrors.163.com
* extras: centos.ustc.edu.cn
* updates: centos.ustc.edu.cn
Warning: No matches found for: glibc-debuginfo
No Matches found
no matches found again.
i try to run yum search glibc
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* addons: centos.ustc.edu.cn
* base: mirrors.163.com
* extras: centos.ustc.edu.cn
* updates: centos.ustc.edu.cn
================================================================================ Matched: glibc =================================================================================
compat-glibc.i386 : Compatibility C library
compat-glibc.x86_64 : Compatibility C library
compat-glibc-headers.x86_64 : Header files for development using standard C libraries.
glibc.i686 : The GNU libc libraries.
glibc.x86_64 : The GNU libc libraries.
glibc-common.x86_64 : Common binaries and locale data for glibc
glibc-devel.i386 : Object files for development using standard C libraries.
glibc-devel.x86_64 : Object files for development using standard C libraries.
glibc-headers.x86_64 : Header files for development using standard C libraries.
glibc-utils.x86_64 : Development utilities from GNU C library
kernel-headers.x86_64 : Header files for the Linux kernel for use by glibc
nss_db.i386 : An NSS library for the Berkeley DB.
nss_db.x86_64 : An NSS library for the Berkeley DB.
yp-tools.x86_64 : NIS (or YP) client programs.
yum-protect-packages.noarch : Yum plugin to prevents Yum from removing itself and other protected packages
i try to run sudo yum install glibc-devel.x86_64, and gdb the coredump file again,
but it display the following
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
how can i find the code line cause coredump? i try to google, but not found, any ideas?

First, although __correctly_grouped_prefixwc caused the segmentation fault, it's likely that it was passed incorrect arguments from some other piece of code, perhaps strtod or strtol or something that called them. That being said, here is how to set things up so that gdb can show the line of source code in __correctly_grouped_prefixwc that caused the segmentation fault.
To do source-level debugging, you need an executable or shared object's debug info, and its source code. Linux and Unix distributions in general do not include these by default, to conserve storage space, but they make them available as packages.
On CentOS, you just need to install the debuginfo package for each executable or library you're interested in. To do this, run
sudo yum install yum-utils
which will install the debuginfo-install program, then run
sudo debuginfo-install glibc
to download and install the glibc-debuginfo-2.5-123 package (your version number may vary). This will install, among many other files, /usr/lib/debug/lib64/libc.so.6.debug, /usr/lib/debug/lib64/libc-2.5.so.debug, and /usr/src/debug/glibc-2.5-20061008T1257/stdlib/grouping.c, which are what you need.
debuginfo-install is a short python program that enables the debuginfo repositories and downloads and installs the debuginfo package corresponding to the package you give as an argument, plus all its dependencies. As an alternative, you can download the debuginfo packages directly from http://debuginfo.centos.org (or any mirrors) and install them using rpm -i.
You mentioned that you got the error No debuginfo packages available to install. Perhaps you don't have the debuginfo repo configured. On my CentOS 5 system, the configuration is in the file /etc/yum.repos.d/CentOS-Debuginfo.repo
# All debug packages from all the various CentOS-5 releases
# are merged into a single repo, split by BaseArch
#
# Note: packages in the debuginfo repo are currently not signed
#
[base-debuginfo]
name=CentOS-5 - Debuginfo
baseurl=http://debuginfo.centos.org/5/$basearch/
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5
enabled=0
For other releases, general instructions for adding the debuginfo repo are in this CentOS wiki article.

Related

How to install libstdc++6 debug symbols on Ubuntu 20.04?

For example, take the following minimal example:
#include <cstdio>
#include <stdexcept>
int main(int argc, char* argv[]){
#ifdef __GLIBCPP__
std::printf("GLIBCPP: %d\n",__GLIBCPP__);
#endif
#ifdef __GLIBCXX__
std::printf("GLIBCXX: %d\n",__GLIBCXX__);
#endif
throw std::runtime_error("Were are libstdc++.so.6 debug symbols?");
return 0;
}
When running it inside my gdb, it does not show the debug symbols for libstdc++.so.6:
$ g++ -o testmain test.cpp -ggdb --std=c++98 && gdb ./testmain
GNU gdb (Ubuntu 9.1-0ubuntu1) 9.1
Copyright (C) 2020 Free Software Foundation, Inc.
...
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./testmain...
(gdb) r
Starting program: /home/user/Downloads/testmain
GLIBCXX: 20200408
terminate called after throwing an instance of 'std::runtime_error'
what(): Were are libstdc++.so.6 debug symbols?
Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt f
#0 __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
set = {__val = {0, 0, 0, 0, 0, 0, 0, 0, 29295, 0, 0, 0, 0, 0, 0, 0}}
pid = <optimized out>
tid = <optimized out>
ret = <optimized out>
#1 0x00007ffff7be1859 in __GI_abort () at abort.c:79
save_stage = 1
act = {__sigaction_handler = {sa_handler = ... <stderr>}
sigs = {__val = {32, 0 <repeats 15 times>}}
#2 0x00007ffff7e67951 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#3 0x00007ffff7e7347c in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#4 0x00007ffff7e734e7 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#5 0x00007ffff7e73799 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#6 0x000055555555524a in main (argc=1, argv=0x7fffffffdef8) at test.cpp:11
No locals.
(gdb)
It just shows No symbol table info available for the libstdc++.so.6 frames.
How can I show the symbols for the libstdc++.so.6?
Searching on this list https://packages.ubuntu.com/search?keywords=libstdc%2B%2B6, I already tried installing the following packages, but none of them fixed the problem:
libgcc-10-dev:amd64 <none> 10.2.0-5ubuntu1~20.0
libstdc++-10-dev:amd64 <none> 10.2.0-5ubuntu1~20.0
libstdc++6-10-dbg:amd64 <none> 10.2.0-5ubuntu1~20.0
libc6-amd64-cross:all <none> 2.31-0ubuntu7cross
linux-libc-dev-amd64-cross:all <none> 5.4.0-21.25cross
libc6-dev-amd64-cross:all <none> 2.31-0ubuntu7cross
libstdc++6-amd64-cross:all <none> 10.2.0-5ubuntu1~20.04cross
libgcc-10-dev-amd64-cross:all <none> 10.2.0-5ubuntu1~20.04cross
libstdc++-10-dev-amd64-cross:all <none> 10.2.0-5ubuntu1~20.04cross
libstdc++6-10-dbg-amd64-cross:all <none> 10.2.0-5ubuntu1~20.04cross
libx32stdc++6-10-dbg:amd64 <none> 10.2.0-5ubuntu1~20.0
Related questions:
How do you find what version of libstdc++ library is installed on your linux machine?
/usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.15' not found
$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"
Update 1
$ dpkg --list | grep libstdc++6
ii libstdc++6:amd64 10.2.0-5ubuntu1~20.04 amd64 GNU Standard C++ Library v3
ii libstdc++6-10-dbg-amd64-cross 10.2.0-5ubuntu1~20.04cross1 all GNU Standard C++ Library v3 (debug build) (amd64)
ii libstdc++6-7-dbg:amd64 7.5.0-6ubuntu2 amd64 GNU Standard C++ Library v3 (debug build)
ii libstdc++6-amd64-cross 10.2.0-5ubuntu1~20.04cross1 all GNU Standard C++ Library v3 (amd64)
Update 2
$ dpkg --list | grep libstdc++6
ii libstdc++6:amd64 10.2.0-5ubuntu1~20.04 amd64 GNU Standard C++ Library v3
ii libstdc++6-10-dbg:amd64 10.2.0-5ubuntu1~20.04 amd64 GNU Standard C++ Library v3 (debug build)
ii libstdc++6-10-dbg-amd64-cross 10.2.0-5ubuntu1~20.04cross1 all GNU Standard C++ Library v3 (debug build) (amd64)
ii libstdc++6-amd64-cross 10.2.0-5ubuntu1~20.04cross1 all GNU Standard C++ Library v3 (amd64)
Background Story:
Days ago, I was also curious about the same question as yours. But that's on CentOS.
What can I do differently after I install those missing debug info packages for gdb?
You can check the question to see what I learnt during searching, I solve your question with those prior knowledge.
In short, for the same thing, in CentOS the difficulties come down to installing the debug info packages. Because the gdb in CentOS tells what exact version of some debug info files you need to install and it gives the full command.
debuginfo-install glibc-2.17-307.el7.1.x86_64 libgcc-4.8.5-44.el7.x86_64 libstdc++-4.8.5-44.el7.x86_64
But this command just can't work and you need to manually add some package sources to install that .
However, as soon as you succeed installing the debug info packages, everything else is set up nicely, even the source files! You can s step into e.g. abort() and list around the source code!
In Ubuntu:
You have to find the exact version of your libstdc++.so.xxx and install the corresponding debug info files.
No libarary(e.g. libstdc++) source files will be installed and set up after install the corresponding debug info files packages. But you can manually do it with set substitute-path.
Answer Part:
I made my gdb work under Ubuntu 18.04.5 LTS. I think that may applies to yours too.
I assume you know this https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html .
So firstly I ldd my.a.out.
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fbfa6f84000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbfa697b000)
...
In my Ubuntu, reading debug symbol for libc.so.6 is successful. So I want to check both .so files' .gnu_debuglink section.
libc.so.6 is a link to libc-2.27.so
so I read the above section with readelf -x.gnu_debuglink libc-2.27.so and gives me:
Hex dump of section '.gnu_debuglink':
0x00000000 6c696263 2d322e32 372e736f 00000000 libc-2.27.so....
0x00000010 32e033a0 2.3.
This means its debug info file's name is libc-2.27.so, which exists in /usr/lib/debug/lib/x86_64-linux-gnu directory.
Now check libstdc++.so.6, which is a link to libstdc++.so.6.0.25 in my machine.
readelf -x.gnu_debuglink libstdc++.so.6.0.25 gives:
Hex dump of section '.gnu_debuglink':
0x00000000 31313961 34346139 39373538 31313436 119a44a997581146
0x00000010 32306338 65396438 65323433 64373039 20c8e9d8e243d709
0x00000020 34663737 66362e64 65627567 00000000 4f77f6.debug....
0x00000030 30573da0 0W=.
This 119a44a99758114620c8e9d8e243d7094f77f6.debug is a build-id debug file.
Learnt from your question and comments below, I do dpkg --list | grep libstdc++ and shows
ii libstdc++-7-dev:amd64 7.5.0-3ubuntu1~18.04 amd64 GNU Standard C++ Library v3 (development files)
ii libstdc++-8-dev:amd64 8.4.0-1ubuntu1~18.04 amd64 GNU Standard C++ Library v3 (development files)
ii libstdc++6:amd64 8.4.0-1ubuntu1~18.04 amd64 GNU Standard C++ Library v3
ii libstdc++6:i386 8.4.0-1ubuntu1~18.04 i386 GNU Standard C++ Library v3
So I sudo apt install libstdc++6-8-dbg.
Then I used dpgk-query -L libstdc++6-8-dbg to see what files are installed with this packages.
tianhe#tianhe-windy:/lib/x86_64-linux-gnu$ dpkg -L libstdc++6-8-dbg
/.
/usr
/usr/lib
/usr/lib/debug
/usr/lib/debug/.build-id
/usr/lib/debug/.build-id/f2
/usr/lib/debug/.build-id/f2/119a44a99758114620c8e9d8e243d7094f77f6.debug
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/debug
/usr/lib/x86_64-linux-gnu/debug/libstdc++.a
/usr/lib/x86_64-linux-gnu/debug/libstdc++.so.6.0.25
/usr/lib/x86_64-linux-gnu/debug/libstdc++fs.a
/usr/share
/usr/share/doc
/usr/share/gdb
/usr/share/gdb/auto-load
/usr/share/gdb/auto-load/usr
/usr/share/gdb/auto-load/usr/lib
/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu
/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/debug
/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/debug/libstdc++.so.6.0.25-gdb.py
/usr/lib/x86_64-linux-gnu/debug/libstdc++.so
/usr/lib/x86_64-linux-gnu/debug/libstdc++.so.6
/usr/share/doc/libstdc++6-8-dbg
And I think I got the debug files when I saw this line:
/usr/lib/debug/.build-id/f2/119a44a99758114620c8e9d8e243d7094f77f6.debug.
Then I open gdb again and it works. I can now s step into string s = "hello";.
So try check what I describe above see if they match.
I followed these instructions https://www.hiroom2.com/ubuntu-2004-dbgsym-en/.
Adding the debug symbols repo:
#!/bin/sh -e
U=http://ddebs.ubuntu.com
C=$(lsb_release -cs)
cat <<EOF | sudo tee /etc/apt/sources.list.d/ddebs.list
deb ${U} ${C} main restricted universe multiverse
#deb ${U} ${C}-security main restricted universe multiverse
deb ${U} ${C}-updates main restricted universe multiverse
deb ${U} ${C}-proposed main restricted universe multiverse
EOF
wget -O - http://ddebs.ubuntu.com/dbgsym-release-key.asc | \
sudo apt-key add -
sudo apt update -y
Then install symbols for libstdc++6
sudo apt-get install libstdc++6-dbgsym
In addtion to #Rick's answer.
In ubuntu 20.04+, you need to install libstdc++6-dbgsym, and before this you need to add debug symbol repo to apt.
To get the source code, you should run apt source libstdc++6, then run ./debian/rules patch as described in debian/README.source.
(Personally I feel installing debug info and source code in ubuntu is much more complex than centOS. I suggest you to use centOS if you just want to have a look into libstdc++'s source code.

Installation of `brew install llvm` leads to "Segmentation fault: 11" on macOS

For some program LDSTORE I need to install llvm on my Mac (macOS 10.13). I do this using brew install llvm. This leads to a Segmentation fault: 11 message when running ldstore, or other (C++ based?) programs.
How can I fix this?
It clearly has to do with llvm, as doing brew uninstall llvm fixes the issue (obvious ldstore won't work in that case).
For what it's worth: I use the native python 2.7.10.
As per suggestion of Stanislav Pankevich I ran lldb ldstore_v11 followed by r, resulting in this:
lldb ldstore_v11
(lldb) target create "ldstore_v11"
Current executable set to 'ldstore_v11' (x86_64).
(lldb) r
Process 15841 launched: '/Users/swvanderlaan/bin/ldstore_v11' (x86_64)
dyld: Library not loaded: /usr/local/opt/libiomp/lib/libiomp5.dylib
Referenced from: /Users/swvanderlaan/bin/ldstore_v11
Reason: image not found
Process 15841 stopped
* thread #1, stop reason = signal SIGABRT
frame #0: 0x0000000100095216 dyld`__abort_with_payload + 10
dyld`__abort_with_payload:
-> 0x100095216 <+10>: jae 0x100095220 ; <+20>
0x100095218 <+12>: movq %rax, %rdi
0x10009521b <+15>: jmp 0x100094a74 ; cerror_nocancel
0x100095220 <+20>: retq
Target 0: (ldstore_v11) stopped.
It's odd that the library is not found, as I clearly added to my bash_profile the following line: export PATH="/usr/local/opt/llvm/bin:$PATH", as per suggestion of the installation messages.
Hope someone can help me debug this.
Thanks,
Sander
P.S. I hope it's clear, I'm not trying to develop anything, I'm just trying to use LDSTORE.
The problem is that this tool is dynamically linked against libiomp5.dylib, which must be present at /usr/local/opt/libiomp/lib/libiomp5.dylib for it to work.
As suggested by Stanislav, download the precompiled binaries from http://releases.llvm.org/5.0.0/clang+llvm-5.0.0-x86_64-apple-darwin.tar.xz. This contains the library you need: ./lib/libiomp5.dylib. You'll have to copy the library into /usr/local/opt/libiomp/lib, which probably won't exist yet.
Once you've done that, you'll be able to run ldstore.

Xcode: how to build for older Intel processors (i5, Core 2 Duo) on i7

My application is crashing when built on a new Apple laptop and then launched on a much older Apple laptop.
The application is built using Xcode 6.4, on OSX 10.9 and 10.10, when using llvm 6.1 and C++11. The SDK is 10.10, the target OSX is 10.7. Optimizations are off.
The crash is very very early on when the C runtime is loading my application binary and initializing the modules.
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 com.MyCompany.MyApplication 0x000000010cd10e7a _GLOBAL__I_a + 10
1 dyld 0x00007fff61fd3ceb ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) + 265
2 dyld 0x00007fff61fd3e78 ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 40
3 dyld 0x00007fff61fd0871 ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int,
This is before any of my application code. The crash does not occur on the build machine (i7 CPU). Crashes occur on i5 and Core 2 Duo machines. I suspect that an extended (CPU specific) instruction is creating the crash on load.
When I use the same Xcode, same llvm, etc to build the application on the Core 2 Duo machine there is no crash.
I am also using homebrew: libmtp, libusb, libusb-compat, cryptopp, curl (with c-ares, openssl, nghttp2), boost. I have specified C++11 where necessary, and have specified --build-bottle. I am statically linking to these libraries.
I have tried to use otool -tV on all libraries, the final binary, etc to find SSE instructions.
I have tried to set the Xcode LLVM build setting "Enable Additional Vector Extensions" to "platform" and "SSE3" to no avail. This is probably because homebrew isn't passing the --universal flag from curl to the building of openssl and it's cryptlib.
I have taken static libraries libcurl.a (CURL), libssl.a (OpenSSL), libcrypto.a (OpenSSL), libz.a (zlib) from the older machine and added them to my repository. Using Xcode to link them into my application solves the problem.
Are there other tools I can should use to narrow down the offending instruction?
Are there other explanations for the crash?
Addendum:
In addition to building the libraries on an older machine, I have also created a proof of concept, minimal, instant crash program that reports a slightly different crash location, but demonstrates the issue:
On an i7 (new Apple computer with new Intel CPU), use homebrew to install:
brew install curl --with-c-ares --with-openssl
Then copy this source into file sse.cpp:
#define CURL_STATICLIB
#include <curl/curl.h>
int main(int argc, const char * argv[]) {
curl_global_init(CURL_GLOBAL_ALL);
return 0;
}
Compile it:
clang++ sse.cpp -c -arch x86_64 -I/usr/local/opt/curl/include
clang++ -o a.out sse.o /usr/local/opt/openssl/lib/libssl.a /usr/local/opt/openssl/lib/libcrypto.a /usr/local/opt/zlib/lib/libz.a /usr/local/opt/curl/lib/libcurl.a /usr/local/opt/c-ares/lib/libcares.a -stdlib=libc++ -framework LDAP
Now move to an older Apple computer with older Intel CPU, and crash it:
./a.out
Crash Report (compressed):
Process: a.out [569]
...
Code Type: X86-64 (Native)
Parent Process: bash [448]
Responsible: Terminal [339]
...
OS Version: Mac OS X 10.10.5 (14F27)
...
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes: 0x0000000000000001, 0x0000000000000000
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 a.out 0x000000010dbdce3f ENGINE_new + 36
1 a.out 0x000000010dbe05e3 ENGINE_load_dynamic + 11
2 a.out 0x000000010dbdf04a ENGINE_load_builtin_engines + 24
3 a.out 0x000000010dc76b36 Curl_ossl_init + 14
4 a.out 0x000000010dc5c2a5 curl_global_init + 114
5 a.out 0x000000010db51d95 main + 37
6 libdyld.dylib 0x00007fff88b735c9 start + 1
Does your code work when you disable compiler optimizations? If not, how about trying an older version of Xcode? It could just be a compiler bug, though I'd hope not! If you can find a working compiler or set of compiler options to check against, you could use LLVM's bugpoint tool to isolate which file is being miscompiled.
The solution appears to involve using:
export HOMEBREW_BUILD_BOTTLE=1
export HOMEBREW_BOTTLE_ARCH=core2
When building the homebrew libraries. Using Intel XED I was able to check the emitted machine code for unsupported instructions:
xed_cmd="/usr/local/bin/xed"
ar -x libcurl.a
parts=(*.o)
for j in "${parts[#]}"; do
chipcheck=$(${xed_cmd} -i ${j} -chip-check ${chipToCheck})
chiperrors=$(echo "${chipcheck}" | grep "# Total Chip Check Errors")
if [[ "$chiperrors" != "# Total Chip Check Errors: 0" ]] ; then
echo ERROR ${libname} ${j} $chiperrors
fi
done

Caffe Compilation Error: gflags.cc' is being linked both statically and dynamically into this executable

I am trying to install caffe following this tutorial
Basically I have the following error when I type the last make command:
me#dl-01:/home/me/caffe-master$ make runtest
.build_release/tools/caffe
caffe: command line brew
usage: caffe command args
commands:
train train or finetune a model
test score a model
device_query show GPU diagnostic information
time benchmark model execution time
Flags from tools/caffe.cpp:
-gpu (Run in GPU mode on given device ID.) type: int32 default: -1
-iterations (The number of iterations to run.) type: int32 default: 50
-model (The model definition protocol buffer text file..) type: string
default: ""
-snapshot (Optional; the snapshot solver state to resume training.)
type: string default: ""
-solver (The solver definition protocol buffer text file.) type: string
default: ""
-weights (Optional; the pretrained weights to initialize finetuning. Cannot
be set simultaneously with snapshot.) type: string default: ""
.build_release/test/test_all.testbin 0 --gtest_shuffle
ERROR: something wrong with flag 'flagfile' in file '/root/glog-0.3.3/gflags-master/src/gflags.cc'. One possibility: file '/root/glog-0.3.3/gflags-master/src/gflags.cc' is being linked both statically and dynamically into this executable.
make: *** [runtest] Error 1
I don't understand how to solve this error. Did anybody find this error before? how can I solve it?
Whether or not you've already solved this somewhere else, I'm posting the answer here in-case others run into the same problem.
Primarily, this problem seems to have come about because we don't always read things properly and blindly follow all instructions thinking they all apply to our case. hint: they don't.
In the installation instructions for Caffe (presuming Ubuntu instructions), there is a section which states:
Everything is packaged in 14.04.
sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler
Blindly ignoring the next title, which states clearly:
Remaining dependencies, 12.04
we go on to install these dependencies, building and installing as required, resulting in the unfortunate side-effect of having 2 versions of libgflags, one dynamic (in /usr/lib[/x86_x64] and one static in /usr/local/lib
Resolution
Promise ourselves failthfully we'll read instructions properly next time around.
Uninstall libgflags
sudo apt-get remove -y libgflags
Delete make install versions
sudo rm -f /usr/local/lib/libgflags.a /usr/local/lib/libgflags_nothreads.a
sudo rm -rf /usr/local/include/gflags
Clean Caffe build
cd <path>/<to>/caffe
make clean
Re-install libgflags package
sudo apt-get install -y libgflags-dev
Rebuild Caffe
make all
make test
make runtest
Et Voila. All tests should now run and you're ready to rock the deep-learning boat.
I've worked out a way to debug this issue analytically. In my case, I was cross-compiling for an older ABI, so apt-get wasn't an option and I was compiling all dependencies manually.
First let's take a look at what this issue actually is. In the Google GFlags library, flags are declared through global objects. When the global object's constructor is run, it calls into the GFlags library to register that command line flag. If the global constructor gets run multiple times (due to multiple versions of the library containing it being loaded into memory), then the GFlags register method dies with an error.
What does GLog have to do with this? Well, GLog uses GFlags, and it has globally declared flag objects. Even if GFlags is linked correctly, if the GLog library gets loaded multiple times, you get an error pointing to logging.cc in GLog.
Sounds like quite a mess, huh. Even if GLog and GFlags are linked as shared in most cases, if another library links to a static version or some other version, kaboom.
Luckily, we can debug this issue using GDB and other tools, if you're willing to delve through some tricky symbol analysis.
First, you'll want to run GDB on the Python interpreter when it tries to import caffe:
gdb --args python -c 'import caffe'
Now, run the program once through so that GDB can pick up all the libraries it imports:
(gdb) r
Now, we can set a breakpoint on the place in the function (FlagRegistry::RegisterFlag()) that prints the error message, and run it again. Note that this line number is from my version of GFlags (2.2.2), you may have to look at the source code of your GFlags version and get the line number.
(gdb) break gflags.c:728
(gdb) r
Hopefully, GDB should then break on the first instance of the error (if not, check that gflags has been built with debugging symbols).
Look at the backtrace:
(gdb) bt
#0 google::(anonymous namespace)::FlagRegistry::RegisterFlag (this=0xa33b30, flag=0x1249d20) at dev/gflags-2.2.2/src/gflags.cc:728
#1 0x00007ffff0f3247a in _GLOBAL__sub_I_logging.cc () from prefix/lib/libcaffe2.so
#2 0x00007ffff7de76ca in call_init (l=<optimized out>, argc=argc#entry=3, argv=argv#entry=0x7fffffffdb08, env=env#entry=0x7fffffffdb28) at dl-init.c:72
#3 0x00007ffff7de77db in call_init (env=0x7fffffffdb28, argv=0x7fffffffdb08, argc=3, l=<optimized out>) at dl-init.c:30
#4 _dl_init (main_map=main_map#entry=0xd9c2a0, argc=3, argv=0x7fffffffdb08, env=0x7fffffffdb28) at dl-init.c:120
#5 0x00007ffff7dec8f2 in dl_open_worker (a=a#entry=0x7fffffffcf70) at dl-open.c:575
#6 0x00007ffff7de7574 in _dl_catch_error (objname=objname#entry=0x7fffffffcf60, errstring=errstring#entry=0x7fffffffcf68, mallocedp=mallocedp#entry=0x7fffffffcf5f,
operate=operate#entry=0x7ffff7dec4e0 <dl_open_worker>, args=args#entry=0x7fffffffcf70) at dl-error.c:187
#7 0x00007ffff7debdb9 in _dl_open (file=0x9aee70 "prefix/lib/python2.7/site-packages/caffe2/python/caffe2_pybind11_state.so", mode=-2147483646,
caller_dlopen=0x51bb39 <_PyImport_GetDynLoadFunc+233>, nsid=-2, argc=<optimized out>, argv=<optimized out>, env=0x7fffffffdb28) at dl-open.c:660
#8 0x00007ffff75ecf09 in dlopen_doit (a=a#entry=0x7fffffffd1a0) at dlopen.c:66
#9 0x00007ffff7de7574 in _dl_catch_error (objname=0xabf9f0, errstring=0xabf9f8, mallocedp=0xabf9e8, operate=0x7ffff75eceb0 <dlopen_doit>, args=0x7fffffffd1a0) at dl-error.c:187
#10 0x00007ffff75ed571 in _dlerror_run (operate=operate#entry=0x7ffff75eceb0 <dlopen_doit>, args=args#entry=0x7fffffffd1a0) at dlerror.c:163
#11 0x00007ffff75ecfa1 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#12 0x000000000051bb39 in _PyImport_GetDynLoadFunc ()
<snip>
Well that's a lot to deal with, but let's focus on the line that's actually important:
#1 0x00007ffff0f3247a in _GLOBAL__sub_I_logging.cc () from prefix/lib/libcaffe2.so
This is the call to the constructor for the global variables in logging.cc (which is part of GLog). As you can see, this call is in libcaffe2.so, meaning that GLog has been statically linked to libcaffe2.so [I was using caffe2, but this procedure should be the same for both].
You can then set a breakpoint on google::(anonymous namespace)::FlagRegistry::RegisterFlag and rerun the program from the start. Look at each call to RegisterFlag(), and figure out where this particular flag was registered the first time. If the library providing the flag is a shared library, then it should only ever get registered from that .so file, and nowhere else.
To confirm the diagnosis, you can use
nm <library> | grep _GLOBAL__sub_I_logging.cc
to check for that init function in a library file. Once you've found your culprit, you'll need to rebuild it so that it doesn't link to GFlags/GLog statically.
I also had two libraries installed, a shared .so library and a static .a library. I removed them all as well as the /usr/local/include/glog folder.
The .so file I had brought over when I (cross) compiled the system, while the .a was from a native and up-to-date build.
Ultimately it came down to building glog (natively) in such a way that it provided the .so files.
I started with a clean download:
git clone git://github.com/google/glog
Then I edited CMakeLists.txt.
Where it says:
add_library (glog
${GLOG_SRCS}
)
I changed it to:
add_library (glog SHARED
${GLOG_SRCS}
)
Next you should be able to follow the other instructions. For my particular case I had to use slightly different instructions, not saying you have to do this. For me it was:
mkdir build
cd build
export CXXFLAGS="-fPIC"
cmake ..
make
sudo make install
This gave me the .so files and put them in the right place. Then I started over with caffe and it fixed the error for me.

gdb fails on mountain lion

I tried to compile a 7.x Version of gdb without any luck.
I codesigned the executable(http://sourceware.org/gdb/wiki/BuildingOnDarwin).
With following version there were these problems.
7.5,7.4,git clone: unknown load command 0x2a (and others) for my application and different system libraries when starting gbd. When trying to print a vector for example i always get:
Could not find the frame base for "main(int, char**)"
7.3 (macports and from gdb-website): on starting the application it fails to set the breakpoint and continues to run.
(gdb) start
Temporary breakpoint 1 at 0x100000950: file ../src/main.cpp, line 15.
Starting program: [...]
BFD: unable to read unknown load command 0x24
BFD: unable to read unknown load command 0x2a
BFD: unable to read unknown load command 0x26
Error in re-setting breakpoint 1: Cannot access memory at address 0x100000950
[application continues]
I used the system llvm-gcc, gcc4.7 and svn-gcc4.8 to compile.
Has anybody succeeded in installing gdb on Mountain Lion?
i installed gdb 7.5 on mountain lion without problems...
those steps might help you: ./configure --prefix=/usr/local
--enable-targets=x86_64-apple-darwin10 --enable-64-bit-bfd --disable-werror --build=x86_64-apple-darwin10 --host=x86_64-apple-darwin10 --target=x86_64-apple-darwin10
then make install gdb.
create your own certificate like discribed here and sign the gdb
http://sourceware.org/gdb/wiki/BuildingOnDarwin
you have to explicitly state: codesign -s gdb-cert /usr/local/gdb
before you sign the gdb make sure you already set your certificate
to trusted. also close the keychain before signing your gdb.
hope this also works for your
I resolved this issue by following the steps tried here: http://coding.derkeiler.com/Archive/Ada/comp.lang.ada/2012-09/msg00305.html
Steps
sudo chgrp procmod /usr/local/bin/gdb
sudo chmod g+s /usr/local/bin/gdb
Edit /System/Library/LaunchDaemons/com.apple.taskgated.plist and add +p argument to taskgated process
Force kill taskgated process (it will restart)
Try again
Other links:
https://blogs.oracle.com/dns/entry/understanding_the_authorization_framework_on
it doesn't seem to work for me... same issue as with the default homebrew settings
v1:src zeph$ brew install gdb
==> Downloading http://ftpmirror.gnu.org/gdb/gdb-7.5.tar.bz2
Already downloaded: /Library/Caches/Homebrew/gdb-7.5.tar.bz2
==> ./configure --prefix=/usr/local/Cellar/gdb/7.5 --with-python=/usr --with-system-readline --enable-targets=x86_64-apple-darwin10 --enable-64-bit-bfd --disable-werror --build=x86_64-apple-darwin10 --hos
==> make
==> make install
==> Caveats
gdb requires special privileges to access Mach ports.
You will need to codesign the binary. For instructions, see:
http://sourceware.org/gdb/wiki/BuildingOnDarwin
==> Summary
/usr/local/Cellar/gdb/7.5: 62 files, 9.3M, built in 119 seconds
v1:src zeph$ codesign -s gdb-cert /usr/local/Cellar/gdb/7.5/bin/gdb
v1:src zeph$ /usr/local/Cellar/gdb/7.5/bin/gdb --args /Users/zeph/tmp/CouchBase/src/install/bin/memcached -d -u root -P /tmp/0libmemcached_memc.pid -t 1 -p 11221 -U 11221 -m 128
GNU gdb (GDB) 7.5
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin10".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
BFD: /Users/zeph/tmp/CouchBase/src/install/bin/memcached: unknown load command 0x29
BFD: /Users/zeph/tmp/CouchBase/src/install/bin/memcached: unknown load command 0x29
Reading symbols from /Users/zeph/tmp/CouchBase/src/install/bin/memcached...done.
(gdb) run
Starting program: /Users/zeph/tmp/CouchBase/src/install/bin/memcached -d -u root -P /tmp/0libmemcached_memc.pid -t 1 -p 11221 -U 11221 -m 128
Unable to find Mach task port for process-id 28755: (os/kern) failure (0x5).
(please check gdb is codesigned - see taskgated(8))
(gdb)
k, I had to re-sign it after having TRUSTED the certificate all across the board
codesign -fs gdb-cert /usr/local/Cellar/gdb/7.5/bin/gdb
There were two issues in the OP's question. The one regarding signing the executable or modifying taskgated and using setgid procmod has been covered. The second issue is the warnings about unknown load commands. I too ran into this and after a bunch of searching came across the following patches, that fix it:
https://gist.github.com/davidbalbert/4197567