The main point of a crash reporter tool like Google breakpad is to generate core dump or minidump files from stripped binaries to process later with debugging symbols. normally these binaries are release builds with compiler optimizations applied and also stripped.
To reproduce the problem on Linux:
1.Build + install google breakpad:
git clone https://chromium.googlesource.com/breakpad/breakpad && cd breakpad
git clone https://chromium.googlesource.com/linux-syscall-support src/third_party/lss
./configure --prefix=/usr/local
make -j$(nproc) && sudo make install
2.The Code:
CMakeLists.txt:
cmake_minimum_required(VERSION 3.10)
project(BreakPadTest)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_VERBOSE_MAKEFILE TRUE)
set(BREAKPAD_DIR "/usr/local/include/breakpad")
option(OPTION_WITH_O1 "With -O1" OFF)
if(OPTION_WITH_O1)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -O1 -fno-omit-frame-pointer")
endif()
include_directories(
${BREAKPAD_DIR}
)
add_executable(${PROJECT_NAME} "main.cc")
target_link_libraries(${PROJECT_NAME}
-lstdc++fs
-pthread
libbreakpad_client.a
)
main.cc:
#include <thread>
#include <experimental/filesystem>
#include <client/linux/handler/exception_handler.h>
namespace breakpad = google_breakpad;
namespace filesystem = std::experimental::filesystem;
static bool DumpCallBack(const breakpad::MinidumpDescriptor& md,
void* context,
bool success) {
(void)md;
(void)context;
return success;
}
static void fault(unsigned after) {
std::this_thread::sleep_for(std::chrono::seconds{after});
delete reinterpret_cast<std::string*>(0xFEE1DEAD);
}
int32_t main(int argc, char** argv) {
(void)argc;
(void)argv;
auto pwd = filesystem::current_path();
const auto dumpDir = pwd.string() + "/dumps";
filesystem::create_directory(dumpDir);
breakpad::MinidumpDescriptor md(dumpDir);
new google_breakpad::ExceptionHandler(
md,
/* FilterCallback */ nullptr,
DumpCallBack,
/* callback_context */ nullptr,
true,
-1
);
fault(1U);
return EXIT_SUCCESS;
}
dump.sh:
#!/bin/bash
#
# e.g ./dump.sh ./exec $PWD/dumps
#
set -e
set -u
DBG_INFO=$(realpath ${1})
DUMPS_DIR=$(realpath ${2:-$PWD/dumps})
DUMP_SYMS=${3:-~/WorkSpace/libraries/breakpad/src/tools/linux/dump_syms/dump_syms}
STAK_WALK=${4:-~/WorkSpace/libraries/breakpad/src/processor/minidump_stackwalk}
#
# Generate debug symbols
#
base=$(basename $DBG_INFO)
$DUMP_SYMS $DBG_INFO > $DUMPS_DIR/$base.sym
#
# Create dump dir structure
#
list=($(head -n1 $DUMPS_DIR/$base.sym))
hash=${list[3]}
mkdir -p $DUMPS_DIR/symbols/$base/$hash
mv $DUMPS_DIR/$base.sym $DUMPS_DIR/symbols/$base/$hash
#
# Produce stack trace
#
RED='\033[0;36m'
NC='\033[0m' # No Color
tree $DUMPS_DIR
for dmp in $DUMPS_DIR/*.dmp ; do
filename=$(basename -- "${dmp}")
filename="${filename%.*}"
echo -e "generating stack trace for -> ${RED}${dmp}${NC}"
$STAK_WALK ${dmp} $DUMPS_DIR/symbols > $DUMPS_DIR/${filename}.txt 2>/dev/null
done
3.Run normal Debug version:
cmake -DCMAKE_BUILD_TYPE=Debug . && make
./BreakPadTest
4.Process minidump generated from stage 3:
./dump.sh ./BreakPadTest ./dumps
stackwalk:
Operating system: Linux
0.0.0 Linux 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64
CPU: amd64
family 6 model 58 stepping 9
1 CPU
GPU: UNKNOWN
Crash reason: SIGSEGV /SEGV_MAPERR
Crash address: 0xfee1dead
Process uptime: not available
Thread 0 (crashed)
0 BreakPadTest!std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_data() const [basic_string.h : 176 + 0x4]
rax = 0x00000000fee1dead rdx = 0x00007ffcfccfcb90
rcx = 0x00007f5b0cbb4bc1 rbx = 0x0000557d577eb8e0
rsi = 0x00007ffcfccfcb90 rdi = 0x00000000fee1dead
rbp = 0x00007ffcfccfcb50 rsp = 0x00007ffcfccfcb50
r8 = 0x0000000000000000 r9 = 0x0000557d577efaf8
r10 = 0xfffffffffffff60b r11 = 0x0000000000000246
r12 = 0x0000557d56d3d2c0 r13 = 0x00007ffcfccfce50
r14 = 0x0000000000000000 r15 = 0x0000000000000000
rip = 0x0000557d56d3dfda
Found by: given as instruction pointer in context
1 BreakPadTest!std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_is_local() const [basic_string.h : 211 + 0xc]
rbx = 0x0000557d577eb8e0 rbp = 0x00007ffcfccfcb80
rsp = 0x00007ffcfccfcb60 r12 = 0x0000557d56d3d2c0
r13 = 0x00007ffcfccfce50 r14 = 0x0000000000000000
r15 = 0x0000000000000000 rip = 0x0000557d56d3e2c5
Found by: call frame info
2 BreakPadTest!std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose() [basic_string.h : 220 + 0xc]
rbx = 0x0000557d577eb8e0 rbp = 0x00007ffcfccfcba0
rsp = 0x00007ffcfccfcb90 r12 = 0x0000557d56d3d2c0
r13 = 0x00007ffcfccfce50 r14 = 0x0000000000000000
r15 = 0x0000000000000000 rip = 0x0000557d56d3dff8
Found by: call frame info
3 BreakPadTest!std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string() [basic_string.h : 657 + 0xc]
rbx = 0x0000557d577eb8e0 rbp = 0x00007ffcfccfcbc0
rsp = 0x00007ffcfccfcbb0 r12 = 0x0000557d56d3d2c0
r13 = 0x00007ffcfccfce50 r14 = 0x0000000000000000
r15 = 0x0000000000000000 rip = 0x0000557d56d3d930
Found by: call frame info
4 BreakPadTest!fault [main.cc : 19 + 0xa]
rbx = 0x0000557d577eb8e0 rbp = 0x00007ffcfccfcbf0
rsp = 0x00007ffcfccfcbd0 r12 = 0x0000557d56d3d2c0
r13 = 0x00007ffcfccfce50 r14 = 0x0000000000000000
r15 = 0x0000000000000000 rip = 0x0000557d56d3d3f0
Found by: call frame info
5 BreakPadTest!main [main.cc : 39 + 0xa]
rbx = 0x0000557d577eb8e0 rbp = 0x00007ffcfccfcd70
rsp = 0x00007ffcfccfcc00 r12 = 0x0000557d56d3d2c0
r13 = 0x00007ffcfccfce50 r14 = 0x0000000000000000
r15 = 0x0000000000000000 rip = 0x0000557d56d3d4fd
Found by: call frame info
6 libc.so.6 + 0x2409b
rbx = 0x0000000000000000 rbp = 0x0000557d56d78b80
rsp = 0x00007ffcfccfcd80 r12 = 0x0000557d56d3d2c0
r13 = 0x00007ffcfccfce50 r14 = 0x0000000000000000
r15 = 0x0000000000000000 rip = 0x00007f5b0ca0609b
Found by: call frame info
7 BreakPadTest!fault [main.cc : 20 + 0x3]
rsp = 0x00007ffcfccfcda0 rip = 0x0000557d56d3d402
Found by: stack scanning
8 ld-linux-x86-64.so.2 + 0xf476
rsp = 0x00007ffcfccfce10 rip = 0x00007f5b0cf1c476
Found by: stack scanning
9 BreakPadTest!_start + 0x2a
rsp = 0x00007ffcfccfce40 rip = 0x0000557d56d3d2ea
Found by: stack scanning
10 0x7ffcfccfce48
rsp = 0x00007ffcfccfce48 rip = 0x00007ffcfccfce48
Found by: call frame info
Loaded modules:
0x557d56d34000 - 0x557d56d78fff BreakPadTest ??? (main)
0x7f5b0c9e2000 - 0x7f5b0cb4bfff libc.so.6 ??? (WARNING: No symbols, libc.so.6, A8A9B91823C5CFE5E5B5D946D605D0920)
0x7f5b0cba3000 - 0x7f5b0cbb7fff libpthread.so.0 ???
0x7f5b0cbc4000 - 0x7f5b0cbd7fff libgcc_s.so.1 ???
0x7f5b0cbde000 - 0x7f5b0cc89fff libm.so.6 ???
0x7f5b0cd61000 - 0x7f5b0ce95fff libstdc++.so.6 ???
0x7f5b0cf0d000 - 0x7f5b0cf2bfff ld-linux-x86-64.so.2 ??? (WARNING: No symbols, ld-linux-x86-64.so.2, 7BFD5DF2BE95A34B86FD71080ACCAE8C0)
0x7ffcfcdc5000 - 0x7ffcfcdc6fff linux-gate.so ???
5.Run stage 3 with -O1:
cmake -DCMAKE_BUILD_TYPE=Debug -DOPTION_WITH_O1=ON . && make
./BreakPadTest
6.Process minidump like stage 4:
stackwalk:
Operating system: Linux
0.0.0 Linux 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64
CPU: amd64
family 6 model 58 stepping 9
1 CPU
GPU: UNKNOWN
Crash reason: SIGSEGV /SEGV_MAPERR
Crash address: 0xfee1dead
Process uptime: not available
Thread 0 (crashed)
0 BreakPadTest!main [basic_string.h : 176 + 0x0]
rax = 0x0000000000000000 rdx = 0x000055bd46f66a40
rcx = 0x00007f7633ea8bc1 rbx = 0x00007ffde4cc7c40
rsi = 0x00007ffde4cc7c40 rdi = 0x00007ffde4cc7c40
rbp = 0x00007ffde4cc7d90 rsp = 0x00007ffde4cc7c20
r8 = 0x0000000000000000 r9 = 0x000055bd474caaf8
r10 = 0x0000000000000000 r11 = 0x0000000000000246
r12 = 0x000055bd474c64f0 r13 = 0x000055bd474c64f0
r14 = 0x0000000000000000 r15 = 0x0000000000000000
rip = 0x000055bd46f1b8dd
Found by: given as instruction pointer in context
1 libc.so.6 + 0x2409b
rbx = 0x0000000000000000 rbp = 0x000055bd46f555e0
rsp = 0x00007ffde4cc7da0 r12 = 0x000055bd46f1b270
r13 = 0x00007ffde4cc7e70 r14 = 0x0000000000000000
r15 = 0x0000000000000000 rip = 0x00007f7633cfa09b
Found by: call frame info
2 BreakPadTest!DumpCallBack [main.cc : 15 + 0x3]
rsp = 0x00007ffde4cc7dc0 rip = 0x000055bd46f1b358
Found by: stack scanning
3 ld-linux-x86-64.so.2 + 0xf476
rsp = 0x00007ffde4cc7e30 rip = 0x00007f7634210476
Found by: stack scanning
4 BreakPadTest!_start + 0x2a
rsp = 0x00007ffde4cc7e60 rip = 0x000055bd46f1b29a
Found by: stack scanning
5 0x7ffde4cc7e68
rsp = 0x00007ffde4cc7e68 rip = 0x00007ffde4cc7e68
Found by: call frame info
Loaded modules:
0x55bd46f14000 - 0x55bd46f55fff BreakPadTest ??? (main)
0x7f7633cd6000 - 0x7f7633e3ffff libc.so.6 ??? (WARNING: No symbols, libc.so.6, A8A9B91823C5CFE5E5B5D946D605D0920)
0x7f7633e97000 - 0x7f7633eabfff libpthread.so.0 ???
0x7f7633eb8000 - 0x7f7633ecbfff libgcc_s.so.1 ???
0x7f7633ed2000 - 0x7f7633f7dfff libm.so.6 ???
0x7f7634055000 - 0x7f7634189fff libstdc++.so.6 ???
0x7f7634201000 - 0x7f763421ffff ld-linux-x86-64.so.2 ??? (WARNING: No symbols, ld-linux-x86-64.so.2, 7BFD5DF2BE95A34B86FD71080ACCAE8C0)
0x7ffde4d9a000 - 0x7ffde4d9bfff linux-gate.so ???
As we can see correct symbols got disappeared from the stack walk of stage 6.
While in other tools like GDB we have correct trace pointing to right location even with -O1 as in stage 5:
Program received signal SIGSEGV, Segmentation fault.
fault (after=1) at /home/iman/WorkSpace/projects/BreakPadTest/src/main.cc:26
26 delete reinterpret_cast<std::string*>(0xFEE1DEAD);
Or in other tools like backward-cpp :
Stack trace (most recent call last):
#3 Object "", at 0xffffffffffffffff, in
#2 Object "/home/iman/WorkSpace/projects/build-CrashReporter-Desktop_Qt_5_11_3_GCC-Debug/CrashReporter", at 0x55f32a66b579, in _start
#1 Source "/build/glibc-vjB4T1/glibc-2.28/csu/../csu/libc-start.c", line 308, in __libc_start_main [0x7f56288be09a]
#0 | Source "/home/iman/WorkSpace/projects/BreakPadTest/src/main.cc", line 48, in main
| 46: #endif // WITH_BREAKPAD
| 47:
| > 48: fault(1U);
| 49:
| 50: return EXIT_SUCCESS;
| Source "/home/iman/WorkSpace/projects/BreakPadTest/src/main.cc", line 26, in fault
| 24: static void fault(unsigned after) {
| 25: std::this_thread::sleep_for(std::chrono::seconds{after});
| > 26: delete reinterpret_cast<std::string*>(0xFEE1DEAD);
| 27: }
| Source "/usr/include/c++/8/bits/basic_string.h", line 657, in
| 655: */
| 656: ~basic_string()
| > 657: { _M_dispose(); }
| 658:
| 659: /**
| Source "/usr/include/c++/8/bits/basic_string.h", line 220, in
| 218: _M_dispose()
| 219: {
| > 220: if (!_M_is_local())
| 221: _M_destroy(_M_allocated_capacity);
| 222: }
| Source "/usr/include/c++/8/bits/basic_string.h", line 211, in
| 209: bool
| 210: _M_is_local() const
| > 211: { return _M_data() == _M_local_data(); }
| 212:
| 213: // Create & Destroy
Source "/usr/include/c++/8/bits/basic_string.h", line 176, in main [0x55f32a66b66c]
174: pointer
175: _M_data() const
> 176: { return _M_dataplus._M_p; }
177:
178: pointer
179: _M_local_data()
Segmentation fault (Address not mapped to object [0xfee1dead])
Segmentation fault
Any idea or hint?
With the optimisation -O1 almost all std::basic_string functions are inline. These functions are inline because std::basic_string is a template and defined in heder files.
There's an old and active issue with google breakpad processor sub-system to handle inline functions or extract inline functions metadata from DWARF debugging information on linux as discussed in these topics:
https://bugzilla.mozilla.org/show_bug.cgi?id=524410
https://bugzilla.mozilla.org/show_bug.cgi?id=563776
https://bugzilla.mozilla.org/show_bug.cgi?id=1665367
https://bugzilla.mozilla.org/show_bug.cgi?id=1398533
https://bugzilla.mozilla.org/show_bug.cgi?id=1636194
https://groups.google.com/g/google-breakpad-discuss/c/ZQOSZBbdF7U/m/58tpSnM1EBsJ
Which requires modification on breakpad internal symbols representation and stack walker but as a workaround you could generate core dump for GDB and get a backtrace with it, for that to happen there's utility inside breakpad code base(src/tools/linux/md2core) called minidump-2-core, so after building your release binary with debugging information:
$ objcopy --only-keep-debug BreakPadTest BreakPadTest.debug
$ strip BreakPadTest
$ ./BreakPadTest
Segmentation fault
$ minidump-2-core -o dumps/45a855b5-8931-4e7e-5f508496-3fe1cacc.core dumps/45a855b5-8931-4e7e-5f508496-3fe1cacc.dmp
$ gdb -c dumps/45a855b5-8931-4e7e-5f508496-3fe1cacc.core BreakPadTest.debug
Program terminated with signal SIGSEGV, Segmentation fault.
#0 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_is_local (this=0xfee1dead) at /usr/include/c++/8/bits/basic_string.h:656
656 ~basic_string()
(gdb) backtrace
#0 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_is_local (this=0xfee1dead) at /usr/include/c++/8/bits/basic_string.h:656
#1 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose (this=0xfee1dead) at /usr/include/c++/8/bits/basic_string.h:220
#2 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string (this=0xfee1dead, __in_chrg=<optimized out>)
at /usr/include/c++/8/bits/basic_string.h:657
#3 fault (after=1) at /home/iman/WorkSpace/projects/BreakPadTest/src/main.cc:19
#4 main (argc=<optimized out>, argv=<optimized out>) at /home/iman/WorkSpace/projects/BreakPadTest/src/main.cc:39
(gdb)
I am looking into the CPP code which i am not very familiar with and facing a crash while assigning a constant simple string in tinyxml2. I have searched for 2 days and could not find any logic for this over the net.
This issue is hardly reproducible.
crash:
pid: 22847, tid: 22967, name: BluetoothAdapte >>> com.android.bluetooth <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
r0 ab425974 r1 e4cf3de5 r2 80000000 r3 ab3f26d8
r4 ab425b00 r5 0000344e r6 00000000 r7 00000000
r8 00000000 r9 ab425bb8 sl 00000004 fp 00000002
ip a0000000 sp e4bd46c8 lr 00000000 pc 00000000 cpsr 800f0010
backtrace:
#00 pc 00000000 <unknown>
stack:
e4bd4688 01000000
e4bd468c f722fc4d /system/lib/libstdc++.so (operator new[](unsigned int)+8)
e4bd4690 ab3f26d8 [heap]
e4bd4694 e4ce76af /system/lib/hw/bluetooth.default.so (tinyxml2::StrPair::SetStr(char const*, int)+40)
e4bd4698 ab425974 [heap]
e4bd469c ab425bd0 [heap]
e4bd46a0 ab425950 [heap]
e4bd46a4 e4cf3de0 /system/lib/hw/bluetooth.default.so
e4bd46a8 e4d3ed20 /system/lib/hw/bluetooth.default.so
e4bd46ac e4ce895b /system/lib/hw/bluetooth.default.so (tinyxml2::XMLElement::FindOrCreateAttribute(char const*)+106)
code :
https://github.com/leethomason/tinyxml2/blob/master/tinyxml2.cpp: crashes at line 177
void StrPair::Reset()
{
if ( flags & NEEDS_DELETE ) {
delete [] start;
}
flags = 0;
start = 0;
end = 0;
}
void StrPair::SetStr( const char* str, int flags )
{
Reset();
size_t len = strlen( str );
start = new char[ len+1 ]; //crashes here
memcpy( start, str, len+1 );
end = start + len;
this->flags = flags | NEEDS_DELETE;
}
I'm writing a C++ class using boost threads. The purpose of the class is to start a new thread, generate in this thread every 2 seconds an object called LocationEstimatePF and call a callback function in another class, giving this LocationEstimatePF as parameter.
Something like this:
void Simulator::startSimulation() {
running_ = true;
sim_thread_ = new boost::thread(boost::bind(&Simulator::run, this));
}
void Simulator::run() {
for (;;) {
if(running_){
try {
//Generate estimate
LocationEstimatePF estimate(1,2,3,4,5);
engine_.updateLocationEstimate(estimate);
//Wait for 2 seconds for next estimate
boost::this_thread::sleep(boost::posix_time::milliseconds(2000));
} catch (boost::thread_interrupted&) {
return;
}
}
}
}
The engine and thread are defined as a member variable in the .h file as follows:
InMapsEngine& engine_;
boost::thread* sim_thread_;
And the callback method in the Engine class:
void updateLocationEstimate(const LocationEstimatePF& estimate) { Log::info("Test");};
Now when I run startSimulation() above, I get a segmentation fault. Now, I know that engine_ is correctly set, I can even call other methods without a problem. I believe the problem lies in the local variable, do you guys have any idea? The "Test" in the engine is never printed.
Here the stack trace:
I/DEBUG (11781): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 00000000
I/DEBUG (11781): r0 00000000 r1 66da3e68 r2 00000001 r3 00000000
I/DEBUG (11781): r4 001e8480 r5 00000000 r6 00000000 r7 40490000
I/DEBUG (11781): r8 62055ab0 r9 66da3e68 sl 3f800000 fp 40000000
I/DEBUG (11781): ip 00000001 sp 66da3e60 lr 61864604 pc 616b0a4e cpsr 60000030
I/DEBUG (11781): d0 4158b82020000000 d1 3f110612c247fff9
I/DEBUG (11781): d2 400b8ae93f57e022 d3 400b8ae943697f55
I/DEBUG (11781): d4 c2ef3a464385c588 d5 407eb7ac40c0a412
I/DEBUG (11781): d6 40dd1ce4bec8a7a3 d7 0062e080400b8ae9
I/DEBUG (11781): d8 3fd9b1b440400000 d9 0000000000000000
I/DEBUG (11781): d10 0000000000000000 d11 0000000000000000
I/DEBUG (11781): d12 0000000000000000 d13 0000000000000000
I/DEBUG (11781): d14 0000000000000000 d15 0000000000000000
I/DEBUG (11781): d16 3fe0000000000000 d17 4158b82000000000
I/DEBUG (11781): d18 3fd7e1cb1c913900 d19 3fc41a88bb99a35a
I/DEBUG (11781): d20 0000000000000000 d21 3fac47489540ee39
I/DEBUG (11781): d22 3f8293cfee13fd75 d23 3f96eb1df9afa780
I/DEBUG (11781): d24 3f5871d465fee876 d25 3f6e1beddd51e95d
I/DEBUG (11781): d26 3f30d636137869f2 d27 3f439d2605c8ef21
I/DEBUG (11781): d28 3fe66819a15509c2 d29 3f13d11c3e64b4a4
I/DEBUG (11781): d30 bef375cbdb605373 d31 3f8b5c41074afda1
I/DEBUG (11781): scr 80000093
I/DEBUG (11781):
I/DEBUG (11781): backtrace:
I/DEBUG (11781): #00 pc 00367a4e /data/app-lib/de.tum.ei.lmt.inmaps3d-1/libInMapsJNI.so (Simulator::run()+69)
I/DEBUG (11781): #01 pc 0051b600 /data/app-lib/de.tum.ei.lmt.inmaps3d-1/libInMapsJNI.so
I/DEBUG (11781):
I/DEBUG (11781): stack:
I/DEBUG (11781): 66da3e20 83a88000
I/DEBUG (11781): 66da3e24 02ed263d
I/DEBUG (11781): 66da3e28 83a88000
I/DEBUG (11781): 66da3e2c 02ed263d
I/DEBUG (11781): 66da3e30 61f05dd3
I/DEBUG (11781): 66da3e34 0004f4cf
I/DEBUG (11781): 66da3e38 532719d5 /dev/ashmem/dalvik-mark-stack (deleted)
I/DEBUG (11781): 66da3e3c 176a6e38
I/DEBUG (11781): 66da3e40 61c0ee99 /data/app-lib/de.tum.ei.lmt.inmaps3d-1/libInMapsJNI.so
I/DEBUG (11781): 66da3e44 001e8480
I/DEBUG (11781): 66da3e48 00000000
I/DEBUG (11781): 66da3e4c 00000000
I/DEBUG (11781): 66da3e50 40490000 /system/lib/libskia.so (GrGLCaps::initStencilFormats(GrGLContextInfo const&)+808)
I/DEBUG (11781): 66da3e54 62055ab0
I/DEBUG (11781): 66da3e58 df0027ad
I/DEBUG (11781): 66da3e5c 00000000
I/DEBUG (11781): #00 66da3e60 001e8480
I/DEBUG (11781): 66da3e64 00000000
I/DEBUG (11781): 66da3e68 00000000
I/DEBUG (11781): 66da3e6c 00000000
I/DEBUG (11781): 66da3e70 00000000
I/DEBUG (11781): 66da3e74 00000000
I/DEBUG (11781): 66da3e78 00000000
I/DEBUG (11781): 66da3e7c 00000000
I/DEBUG (11781): 66da3e80 3f800000
I/DEBUG (11781): 66da3e84 40000000
I/DEBUG (11781): 66da3e88 40400000 /system/lib/libskia.so
I/DEBUG (11781): 66da3e8c 00000000
I/DEBUG (11781): 66da3e90 00000000
I/DEBUG (11781): 66da3e94 40490000 /system/lib/libskia.so (GrGLCaps::initStencilFormats(GrGLContextInfo const&)+808)
I/DEBUG (11781): 66da3e98 00000000
I/DEBUG (11781): 66da3e9c 16c16c17
Your InMapsEngine reference is not initialized or do you set it up outside of the class constructor ?
InMapsEngine & engine_;
You could use member object instead
InMapsEngine engine_;
... or set your reference before the thread calls it.
It's hard to tell from your stacktrace exactly what is going wrong, but it could be that you have multiple threads trying to access your shared engine resource at the same time.
One way to do this would be to use a mutex and condition variable to signal to your threads when your engine resource is available to them.:
try{
boost::mutex::scoped_lock lock(engine_mutex);
m_condition.wait(lock);
engine_.updateLocationEstimate(estimate);
m_condition.notify_one();
}
catch (boost::thread_interrupted&) {
return;
}
boost::this_thread::sleep(boost::posix_time::milliseconds(2000));
Though you may want to use a regular lock instead if you want to include your sleep in the try block. Just remember to unlock.
I am using Stackdumps with Win32, to write all return adresses into my logfile. I match these with a mapfile later on (see my article [Post Mortem Debugging][1]).
EDIT:: Problem solved - see my own answer below.
With Windows x64 i do not find a reliable way to write only the return adresses into the logfile. I tried several ways:
Trial 1: Pointer Arithmetic:
CONTEXT Context;
RtlCaptureContext(&Context);
char *eNextBP = (char *)Context.Rdi;
for(ULONG Frame = 0; eNextBP ; Frame++)
{
char *pBP = eNextBP;
eNextBP = *(char **)pBP; // Next BP in Stack
fprintf(LogFile, "*** %2d called from %016LX (pBP at %016LX)\n", Frame,
(ULONG64)*(char **)(pBP + 8), (ULONG64)pBP);
}
This works fine in the debug version - but it crashes in the release version. The value of Context.Rdi has no usable value there. I did check for differences in the compiler settings (visual Studio 2005). I have not found anything suspicious.
Trial 2: Using StackWalk64
RtlCaptureContext(&Context);
STACKFRAME64 stk;
memset(&stk, 0, sizeof(stk));
stk.AddrPC.Offset = Context.Rip;
stk.AddrPC.Mode = AddrModeFlat;
stk.AddrStack.Offset = Context.Rsp;
stk.AddrStack.Mode = AddrModeFlat;
stk.AddrFrame.Offset = Context.Rbp;
stk.AddrFrame.Mode = AddrModeFlat;
for(ULONG Frame = 0; ; Frame++)
{
BOOL result = StackWalk64(
IMAGE_FILE_MACHINE_AMD64, // __in DWORD MachineType,
GetCurrentProcess(), // __in HANDLE hProcess,
GetCurrentThread(), // __in HANDLE hThread,
&stk, // __inout LP STACKFRAME64 StackFrame,
&Context, // __inout PVOID ContextRecord,
NULL, // __in_opt PREAD_PROCESS_MEMORY_ROUTINE64 ReadMemoryRoutine,
SymFunctionTableAccess64, // __in_opt PFUNCTION_TABLE_ACCESS_ROUTINE64 FunctionTableAccessRoutine,
SymGetModuleBase64, // __in_opt PGET_MODULE_BASE_ROUTINE64 GetModuleBaseRoutine,
NULL // __in_opt PTRANSLATE_ADDRESS_ROUTINE64 TranslateAddress
);
fprintf(gApplSetup.TraceFile, "*** %2d called from %016LX STACK %016LX FRAME %016LX\n", Frame, (ULONG64)stk.AddrPC.Offset, (ULONG64)stk.AddrStack.Offset, (ULONG64)stk.AddrFrame.Offset);
if(! result)
break;
}
This does not only dump the return addresses, but the WHOLE STACK. I receive about 1000 lines in my log file using this approach. I can use this, but i have to search trough the lines and some data of the stacks happens to be a valid code address.
Trial 3: Using Backtrace
static USHORT (WINAPI
*s_pfnCaptureStackBackTrace)(ULONG, ULONG, PVOID*, PULONG) = 0;
if (s_pfnCaptureStackBackTrace == 0)
{
const HMODULE hNtDll = ::GetModuleHandle("ntdll.dll");
reinterpret_cast<void*&>(s_pfnCaptureStackBackTrace)
= ::GetProcAddress(hNtDll, "RtlCaptureStackBackTrace");
}
PVOID myFrames[128];
s_pfnCaptureStackBackTrace(0, 128, myFrames, NULL);
for(int ndx = 0; ndx < 128; ndx++)
fprintf(gApplSetup.TraceFile, "*** BackTrace %3d %016LX\n", ndx, (ULONG64)myFrames[ndx]);
Results in no usable information.
Has anyone implemented such a stack walk in x64 that does only write out the return adresses in the stack? Ive seen the approaches [StackTrace64][2], [StackWalker][3] and other ones. They either do not compile or they are much too much comlicated. It basically is a simple task!
Sample StackDump64.cpp
#include <Windows.h>
#include <DbgHelp.h>
#include <Winbase.h>
#include <stdio.h>
void WriteStackDump()
{
FILE *myFile = fopen("StackDump64.log", "w+t");
CONTEXT Context;
memset(&Context, 0, sizeof(Context));
RtlCaptureContext(&Context);
RtlCaptureContext(&Context);
STACKFRAME64 stk;
memset(&stk, 0, sizeof(stk));
stk.AddrPC.Offset = Context.Rip;
stk.AddrPC.Mode = AddrModeFlat;
stk.AddrStack.Offset = Context.Rsp;
stk.AddrStack.Mode = AddrModeFlat;
stk.AddrFrame.Offset = Context.Rbp;
stk.AddrFrame.Mode = AddrModeFlat;
for(ULONG Frame = 0; ; Frame++)
{
BOOL result = StackWalk64(
IMAGE_FILE_MACHINE_AMD64, // __in DWORD MachineType,
GetCurrentProcess(), // __in HANDLE hProcess,
GetCurrentThread(), // __in HANDLE hThread,
&stk, // __inout LP STACKFRAME64 StackFrame,
&Context, // __inout PVOID ContextRecord,
NULL, // __in_opt PREAD_PROCESS_MEMORY_ROUTINE64 ReadMemoryRoutine,
SymFunctionTableAccess64, // __in_opt PFUNCTION_TABLE_ACCESS_ROUTINE64 FunctionTableAccessRoutine,
SymGetModuleBase64, // __in_opt PGET_MODULE_BASE_ROUTINE64 GetModuleBaseRoutine,
NULL // __in_opt PTRANSLATE_ADDRESS_ROUTINE64 TranslateAddress
);
fprintf(myFile, "*** %2d called from %016I64LX STACK %016I64LX AddrReturn %016I64LX\n", Frame, stk.AddrPC.Offset, stk.AddrStack.Offset, stk.AddrReturn.Offset);
if(! result)
break;
}
fclose(myFile);
}
void funcC()
{
WriteStackDump();
}
void funcB()
{
funcC();
}
void funcA()
{
funcB();
}
int main(int argc, char *argv[])
{
funcA();
}
Running this sample results in the follwing log file content:
*** 0 called from 000000014000109E STACK 000000000012F780 AddrReturn 0000000140005798
*** 1 called from 000000001033D160 STACK 000000000012F788 AddrReturn 00000001400057B0
*** 2 called from 00000001400057B0 STACK 000000000012F790 AddrReturn 0000000000000001
*** 3 called from 0000000000000002 STACK 000000000012F798 AddrReturn 00000001400057B0
*** 4 called from 0000000000000002 STACK 000000000012F7A0 AddrReturn 000000000012F7F0
*** 5 called from 000000000012F7F0 STACK 000000000012F7A8 AddrReturn 0000000000000000
*** 6 called from 0000000000000000 STACK 000000000012F7B0 AddrReturn 000007FF7250CF40
*** 7 called from 000007FF7250CF40 STACK 000000000012F7B8 AddrReturn 000007FF7250D390
*** 8 called from 000007FF7250D390 STACK 000000000012F7C0 AddrReturn 000007FF725B6950
*** 9 called from 000007FF725B6950 STACK 000000000012F7C8 AddrReturn CCCCCCCCCCCCCCCC
*** 10 called from CCCCCCCCCCCCCCCC STACK 000000000012F7D0 AddrReturn 000000001033D160
*** 11 called from 000000001033D160 STACK 000000000012F7D8 AddrReturn CCCCCCCCCCCCCCCC
*** 12 called from CCCCCCCCCCCCCCCC STACK 000000000012F7E0 AddrReturn CCCCCCCCCCCCCCCC
*** 13 called from CCCCCCCCCCCCCCCC STACK 000000000012F7E8 AddrReturn CCCCCCCCCCCCCCCC
*** 14 called from CCCCCCCCCCCCCCCC STACK 000000000012F7F0 AddrReturn 0000000000000000
*** 15 called from 0000000000000000 STACK 000000000012F7F8 AddrReturn 0000000000000000
*** 16 called from 0000000000000000 STACK 000000000012F800 AddrReturn 0000000000000000
*** 17 called from 0000000000000000 STACK 000000000012F808 AddrReturn 0000000000000000
*** 18 called from 0000000000000000 STACK 000000000012F810 AddrReturn 0000000000000000
*** 19 called from 0000000000000000 STACK 000000000012F818 AddrReturn 0000000000000000
*** 20 called from 0000000000000000 STACK 000000000012F820 AddrReturn 00001F800010000F
*** 21 called from 00001F800010000F STACK 000000000012F828 AddrReturn 0053002B002B0033
*** 22 called from 0053002B002B0033 STACK 000000000012F830 AddrReturn 00000206002B002B
*** 23 called from 00000206002B002B STACK 000000000012F838 AddrReturn 0000000000000000
*** 24 called from 0000000000000000 STACK 000000000012F840 AddrReturn 0000000000000000
*** 25 called from 0000000000000000 STACK 000000000012F848 AddrReturn 0000000000000000
*** 26 called from 0000000000000000 STACK 000000000012F850 AddrReturn 0000000000000000
*** 27 called from 0000000000000000 STACK 000000000012F858 AddrReturn 0000000000000000
*** 28 called from 0000000000000000 STACK 000000000012F860 AddrReturn 0000000000000000
*** 29 called from 0000000000000000 STACK 000000000012F868 AddrReturn 0000000000000246
*** 30 called from 0000000000000246 STACK 000000000012F870 AddrReturn 000000000012F7F0
*** 31 called from 000000000012F7F0 STACK 000000000012F878 AddrReturn 0000000000000000
*** 32 called from 0000000000000000 STACK 000000000012F880 AddrReturn 0000000000000000
*** 33 called from 0000000000000000 STACK 000000000012F888 AddrReturn 000000000012F888
*** 34 called from 000000000012F888 STACK 000000000012F890 AddrReturn 0000000000000000
*** 35 called from 0000000000000000 STACK 000000000012F898 AddrReturn 0000000000000000
*** 36 called from 0000000000000000 STACK 000000000012F8A0 AddrReturn 000000000012FE10
*** 37 called from 000000000012FE10 STACK 000000000012F8A8 AddrReturn 0000000000000000
*** 38 called from 0000000000000000 STACK 000000000012F8B0 AddrReturn 0000000000000000
*** 39 called from 0000000000000000 STACK 000000000012F8B8 AddrReturn 0000000000000000
*** 40 called from 0000000000000000 STACK 000000000012F8C0 AddrReturn 0000000000000246
*** 41 called from 0000000000000246 STACK 000000000012F8C8 AddrReturn 0000000000000000
*** 42 called from 0000000000000000 STACK 000000000012F8D0 AddrReturn 0000000000000000
*** 43 called from 0000000000000000 STACK 000000000012F8D8 AddrReturn 0000000000000000
*** 44 called from 0000000000000000 STACK 000000000012F8E0 AddrReturn 0000000000000000
*** 45 called from 0000000000000000 STACK 000000000012F8E8 AddrReturn 0000000000000000
*** 46 called from 0000000000000000 STACK 000000000012F8F0 AddrReturn 000000000000027F
*** 47 called from 000000000000027F STACK 000000000012F8F8 AddrReturn 0000000000000000
*** 48 called from 0000000000000000 STACK 000000000012F900 AddrReturn 0000000000000000
*** 49 called from 0000000000000000 STACK 000000000012F908 AddrReturn 0000FFFF00001F80
*** 50 called from 0000FFFF00001F80 STACK 000000000012F910 AddrReturn 0000000000000000
*** 51 called from 0000000000000000 STACK 000000000012F918 AddrReturn 0000000000000000
*** 52 called from 0000000000000000 STACK 000000000012F920 AddrReturn 0000000000000000
*** 53 called from 0000000000000000 STACK 000000000012F928 AddrReturn 0000000000000000
*** 54 called from 0000000000000000 STACK 000000000012F930 AddrReturn 0000000000000000
*** 55 called from 0000000000000000 STACK 000000000012F938 AddrReturn 0000000000000000
*** 56 called from 0000000000000000 STACK 000000000012F940 AddrReturn 0000000000000000
*** 57 called from 0000000000000000 STACK 000000000012F948 AddrReturn 0000000000000000
*** 58 called from 0000000000000000 STACK 000000000012F950 AddrReturn 0000000000000000
*** 59 called from 0000000000000000 STACK 000000000012F958 AddrReturn 0000000000000000
*** 60 called from 0000000000000000 STACK 000000000012F960 AddrReturn 0000000000000000
*** 61 called from 0000000000000000 STACK 000000000012F968 AddrReturn 0000000000000000
*** 62 called from 0000000000000000 STACK 000000000012F970 AddrReturn 0000000000000000
*** 63 called from 0000000000000000 STACK 000000000012F978 AddrReturn 0000000000000000
*** 64 called from 0000000000000000 STACK 000000000012F980 AddrReturn 0000000000000000
*** 65 called from 0000000000000000 STACK 000000000012F988 AddrReturn 0000000000000000
*** 66 called from 0000000000000000 STACK 000000000012F990 AddrReturn 0000000000000000
*** 67 called from 0000000000000000 STACK 000000000012F998 AddrReturn 0000000000000000
*** 68 called from 0000000000000000 STACK 000000000012F9A0 AddrReturn 0000000000000000
*** 69 called from 0000000000000000 STACK 000000000012F9A8 AddrReturn 0000000000000000
*** 70 called from 0000000000000000 STACK 000000000012F9B0 AddrReturn 0000000000000000
*** 71 called from 0000000000000000 STACK 000000000012F9B8 AddrReturn 0000000000000000
*** 72 called from 0000000000000000 STACK 000000000012F9C0 AddrReturn 0000000000000000
*** 73 called from 0000000000000000 STACK 000000000012F9C8 AddrReturn 0000000000000000
*** 74 called from 0000000000000000 STACK 000000000012F9D0 AddrReturn 0000000000000000
*** 75 called from 0000000000000000 STACK 000000000012F9D8 AddrReturn 0000000000000000
*** 76 called from 0000000000000000 STACK 000000000012F9E0 AddrReturn 0000000000000000
*** 77 called from 0000000000000000 STACK 000000000012F9E8 AddrReturn 0000000000000000
*** 78 called from 0000000000000000 STACK 000000000012F9F0 AddrReturn 0000000000000000
*** 79 called from 0000000000000000 STACK 000000000012F9F8 AddrReturn 0000000000000000
*** 80 called from 0000000000000000 STACK 000000000012FA00 AddrReturn 0000000000000000
*** 81 called from 0000000000000000 STACK 000000000012FA08 AddrReturn 0000000000000000
*** 82 called from 0000000000000000 STACK 000000000012FA10 AddrReturn 0000000000000000
*** 83 called from 0000000000000000 STACK 000000000012FA18 AddrReturn 0000000000000000
*** 84 called from 0000000000000000 STACK 000000000012FA20 AddrReturn 0000000000000000
*** 85 called from 0000000000000000 STACK 000000000012FA28 AddrReturn 0000000000000000
*** 86 called from 0000000000000000 STACK 000000000012FA30 AddrReturn 0000000000000000
*** 87 called from 0000000000000000 STACK 000000000012FA38 AddrReturn 0000000000000000
*** 88 called from 0000000000000000 STACK 000000000012FA40 AddrReturn 0000000000000000
*** 89 called from 0000000000000000 STACK 000000000012FA48 AddrReturn 0000000000000000
*** 90 called from 0000000000000000 STACK 000000000012FA50 AddrReturn 0000000000000000
*** 91 called from 0000000000000000 STACK 000000000012FA58 AddrReturn 0000000000000000
*** 92 called from 0000000000000000 STACK 000000000012FA60 AddrReturn 0000000000000000
*** 93 called from 0000000000000000 STACK 000000000012FA68 AddrReturn 0000000000000000
*** 94 called from 0000000000000000 STACK 000000000012FA70 AddrReturn 0000000000000000
*** 95 called from 0000000000000000 STACK 000000000012FA78 AddrReturn 0000000000000000
*** 96 called from 0000000000000000 STACK 000000000012FA80 AddrReturn 0000000000000000
*** 97 called from 0000000000000000 STACK 000000000012FA88 AddrReturn 0000000000000000
*** 98 called from 0000000000000000 STACK 000000000012FA90 AddrReturn 0000000000000000
*** 99 called from 0000000000000000 STACK 000000000012FA98 AddrReturn 0000000000000000
*** 100 called from 0000000000000000 STACK 000000000012FAA0 AddrReturn 0000000000000000
*** 101 called from 0000000000000000 STACK 000000000012FAA8 AddrReturn 0000000000000000
*** 102 called from 0000000000000000 STACK 000000000012FAB0 AddrReturn 0000000000000000
*** 103 called from 0000000000000000 STACK 000000000012FAB8 AddrReturn 0000000000000000
*** 104 called from 0000000000000000 STACK 000000000012FAC0 AddrReturn 0000000000000000
*** 105 called from 0000000000000000 STACK 000000000012FAC8 AddrReturn 0000000000000000
*** 106 called from 0000000000000000 STACK 000000000012FAD0 AddrReturn 0000000000000000
*** 107 called from 0000000000000000 STACK 000000000012FAD8 AddrReturn 0000000000000000
*** 108 called from 0000000000000000 STACK 000000000012FAE0 AddrReturn 0000000000000000
*** 109 called from 0000000000000000 STACK 000000000012FAE8 AddrReturn 0000000000000000
*** 110 called from 0000000000000000 STACK 000000000012FAF0 AddrReturn 0000000000000000
*** 111 called from 0000000000000000 STACK 000000000012FAF8 AddrReturn 0000000000000000
*** 112 called from 0000000000000000 STACK 000000000012FB00 AddrReturn 0000000000000000
*** 113 called from 0000000000000000 STACK 000000000012FB08 AddrReturn 0000000000000000
*** 114 called from 0000000000000000 STACK 000000000012FB10 AddrReturn 0000000000000000
*** 115 called from 0000000000000000 STACK 000000000012FB18 AddrReturn 0000000000000000
*** 116 called from 0000000000000000 STACK 000000000012FB20 AddrReturn 0000000000000000
*** 117 called from 0000000000000000 STACK 000000000012FB28 AddrReturn 0000000000000000
*** 118 called from 0000000000000000 STACK 000000000012FB30 AddrReturn 0000000000000000
*** 119 called from 0000000000000000 STACK 000000000012FB38 AddrReturn 0000000000000000
*** 120 called from 0000000000000000 STACK 000000000012FB40 AddrReturn 0000000000000000
*** 121 called from 0000000000000000 STACK 000000000012FB48 AddrReturn 0000000000000000
*** 122 called from 0000000000000000 STACK 000000000012FB50 AddrReturn 0000000000000000
*** 123 called from 0000000000000000 STACK 000000000012FB58 AddrReturn 0000000000000000
*** 124 called from 0000000000000000 STACK 000000000012FB60 AddrReturn 0000000000000000
*** 125 called from 0000000000000000 STACK 000000000012FB68 AddrReturn 0000000000000000
*** 126 called from 0000000000000000 STACK 000000000012FB70 AddrReturn 0000000000000000
*** 127 called from 0000000000000000 STACK 000000000012FB78 AddrReturn 0000000000000000
*** 128 called from 0000000000000000 STACK 000000000012FB80 AddrReturn 0000000000000000
*** 129 called from 0000000000000000 STACK 000000000012FB88 AddrReturn 0000000000000000
*** 130 called from 0000000000000000 STACK 000000000012FB90 AddrReturn 0000000000000000
*** 131 called from 0000000000000000 STACK 000000000012FB98 AddrReturn 0000000000000000
*** 132 called from 0000000000000000 STACK 000000000012FBA0 AddrReturn 0000000000000000
*** 133 called from 0000000000000000 STACK 000000000012FBA8 AddrReturn 0000000000000000
*** 134 called from 0000000000000000 STACK 000000000012FBB0 AddrReturn 0000000000000000
*** 135 called from 0000000000000000 STACK 000000000012FBB8 AddrReturn 0000000000000000
*** 136 called from 0000000000000000 STACK 000000000012FBC0 AddrReturn 0000000000000000
*** 137 called from 0000000000000000 STACK 000000000012FBC8 AddrReturn 0000000000000000
*** 138 called from 0000000000000000 STACK 000000000012FBD0 AddrReturn 0000000000000000
*** 139 called from 0000000000000000 STACK 000000000012FBD8 AddrReturn 0000000000000000
*** 140 called from 0000000000000000 STACK 000000000012FBE0 AddrReturn 0000000000000000
*** 141 called from 0000000000000000 STACK 000000000012FBE8 AddrReturn 0000000000000000
*** 142 called from 0000000000000000 STACK 000000000012FBF0 AddrReturn 0000000000000000
*** 143 called from 0000000000000000 STACK 000000000012FBF8 AddrReturn 0000000000000000
*** 144 called from 0000000000000000 STACK 000000000012FC00 AddrReturn 0000000000000000
*** 145 called from 0000000000000000 STACK 000000000012FC08 AddrReturn 0000000000000000
*** 146 called from 0000000000000000 STACK 000000000012FC10 AddrReturn 0000000000000000
*** 147 called from 0000000000000000 STACK 000000000012FC18 AddrReturn 0000000000000000
*** 148 called from 0000000000000000 STACK 000000000012FC20 AddrReturn 0000000000000000
*** 149 called from 0000000000000000 STACK 000000000012FC28 AddrReturn 0000000000000000
*** 150 called from 0000000000000000 STACK 000000000012FC30 AddrReturn 0000000000000000
*** 151 called from 0000000000000000 STACK 000000000012FC38 AddrReturn 0000000000000000
*** 152 called from 0000000000000000 STACK 000000000012FC40 AddrReturn 0000000000000000
*** 153 called from 0000000000000000 STACK 000000000012FC48 AddrReturn 0000000000000000
*** 154 called from 0000000000000000 STACK 000000000012FC50 AddrReturn 0000000000000000
*** 155 called from 0000000000000000 STACK 000000000012FC58 AddrReturn 0000000000000000
*** 156 called from 0000000000000000 STACK 000000000012FC60 AddrReturn 0000000000000000
*** 157 called from 0000000000000000 STACK 000000000012FC68 AddrReturn 0000000000000000
*** 158 called from 0000000000000000 STACK 000000000012FC70 AddrReturn 0000000000000000
*** 159 called from 0000000000000000 STACK 000000000012FC78 AddrReturn 0000000000000000
*** 160 called from 0000000000000000 STACK 000000000012FC80 AddrReturn 0000000000000000
*** 161 called from 0000000000000000 STACK 000000000012FC88 AddrReturn 0000000000000000
*** 162 called from 0000000000000000 STACK 000000000012FC90 AddrReturn 0000000000000000
*** 163 called from 0000000000000000 STACK 000000000012FC98 AddrReturn 0000000000000000
*** 164 called from 0000000000000000 STACK 000000000012FCA0 AddrReturn 0000000000000000
*** 165 called from 0000000000000000 STACK 000000000012FCA8 AddrReturn 0000000000000000
*** 166 called from 0000000000000000 STACK 000000000012FCB0 AddrReturn 0000000000000000
*** 167 called from 0000000000000000 STACK 000000000012FCB8 AddrReturn 0000000000000000
*** 168 called from 0000000000000000 STACK 000000000012FCC0 AddrReturn CCCCCCCCCCCCCCCC
*** 169 called from CCCCCCCCCCCCCCCC STACK 000000000012FCC8 AddrReturn CCCCCCCCCCCCCCCC
*** 170 called from CCCCCCCCCCCCCCCC STACK 000000000012FCD0 AddrReturn CCCCCCCCCCCCCCCC
*** 171 called from CCCCCCCCCCCCCCCC STACK 000000000012FCD8 AddrReturn CCCCCCCCCCCCCCCC
*** 172 called from CCCCCCCCCCCCCCCC STACK 000000000012FCE0 AddrReturn CCCCCCCCCCCCCCCC
*** 173 called from CCCCCCCCCCCCCCCC STACK 000000000012FCE8 AddrReturn 0000000300000000
*** 174 called from 0000000300000000 STACK 000000000012FCF0 AddrReturn 0000000300000000
*** 175 called from 0000000300000000 STACK 000000000012FCF8 AddrReturn 0000000300000000
*** 176 called from 0000000300000000 STACK 000000000012FD00 AddrReturn 000000000012FCF0
*** 177 called from 000000000012FCF8 STACK 000000000012FD08 AddrReturn 0000000300000000
*** 178 called from 0000000300000000 STACK 000000000012FD10 AddrReturn 000000000012FD10
*** 179 called from 000000000012FD18 STACK 000000000012FD18 AddrReturn 0000000300000000
*** 180 called from 0000000300000000 STACK 000000000012FD20 AddrReturn 0000000000000000
*** 181 called from 0000000000000000 STACK 000000000012FD28 AddrReturn 0000000000000000
*** 182 called from 0000000000000000 STACK 000000000012FD30 AddrReturn 0000000000000000
*** 183 called from 0000000000000000 STACK 000000000012FD38 AddrReturn 0000000000000000
*** 184 called from 0000000000000000 STACK 000000000012FD40 AddrReturn 0000000000000000
*** 185 called from 0000000100000000 STACK 000000000012FD48 AddrReturn 0000000100000000
*** 186 called from 0000000000000000 STACK 000000000012FD50 AddrReturn 0000000000000000
*** 187 called from 0000000000000000 STACK 000000000012FD58 AddrReturn 0000000100000000
*** 188 called from 0000000100000000 STACK 000000000012FD60 AddrReturn 0000000000000000
*** 189 called from 0000000000000000 STACK 000000000012FD68 AddrReturn 0000000000000000
*** 190 called from 0000000000000000 STACK 000000000012FD70 AddrReturn 0000000000000000
*** 191 called from 0000000000000000 STACK 000000000012FD78 AddrReturn 0000000000000000
*** 192 called from 0000000000000000 STACK 000000000012FD80 AddrReturn 0000000000000000
*** 193 called from 0000000000000000 STACK 000000000012FD88 AddrReturn 0000000000000000
*** 194 called from 0000000000000000 STACK 000000000012FD90 AddrReturn 0000000000000000
*** 195 called from 0000000000000000 STACK 000000000012FD98 AddrReturn 0000000000000000
*** 196 called from 0000000000000000 STACK 000000000012FDA0 AddrReturn 0000000000000000
*** 197 called from 0000000000000000 STACK 000000000012FDA8 AddrReturn 0000000000000000
*** 198 called from 0000000000000000 STACK 000000000012FDB0 AddrReturn 0000000000000000
*** 199 called from 0000000000000000 STACK 000000000012FDB8 AddrReturn 0000000000000000
*** 200 called from 0000000000000000 STACK 000000000012FDC0 AddrReturn 0000000000000000
*** 201 called from 0000000000000000 STACK 000000000012FDC8 AddrReturn 0000000000000000
*** 202 called from 0000000000000000 STACK 000000000012FDD0 AddrReturn 0000000000000000
*** 203 called from 0000000000000000 STACK 000000000012FDD8 AddrReturn 0000000000000000
*** 204 called from 0000000000000000 STACK 000000000012FDE0 AddrReturn 0000000000000000
*** 205 called from 0000000000000000 STACK 000000000012FDE8 AddrReturn CCCCCCCCCCCCCCCC
*** 206 called from CCCCCCCCCCCCCCCC STACK 000000000012FDF0 AddrReturn 000000CECCCCCCCC
*** 207 called from 000000CFCCCCCCCC STACK 000000000012FDF8 AddrReturn CCCCCCCC00000001
*** 208 called from CCCCCCCC00000001 STACK 000000000012FE00 AddrReturn FFFFFFFFFFFFFFFE
*** 209 called from FFFFFFFFFFFFFFFE STACK 000000000012FE08 AddrReturn CCCCCCCCCCCCCCCC
*** 210 called from CCCCCCCCCCCCCCCC STACK 000000000012FE10 AddrReturn 000000000012FE40
*** 211 called from 000000000012FE40 STACK 000000000012FE18 AddrReturn 000000014000122F
*** 212 called from 000000014000122F STACK 000000000012FE20 AddrReturn CCCCCCCCCCCCCCCC
*** 213 called from CCCCCCCCCCCCCCCC STACK 000000000012FE28 AddrReturn CCCCCCCCCCCCCCCC
*** 214 called from CCCCCCCCCCCCCCCC STACK 000000000012FE30 AddrReturn CCCCCCCCCCCCCCCC
*** 215 called from CCCCCCCCCCCCCCCC STACK 000000000012FE38 AddrReturn CCCCCCCCCCCCCCCC
*** 216 called from CCCCCCCCCCCCCCCC STACK 000000000012FE40 AddrReturn 000000000012FE70
*** 217 called from 000000000012FE70 STACK 000000000012FE48 AddrReturn 000000014000125F
*** 218 called from 000000014000125F STACK 000000000012FE50 AddrReturn CCCCCCCCCCCCCCCC
*** 219 called from CCCCCCCCCCCCCCCC STACK 000000000012FE58 AddrReturn CCCCCCCCCCCCCCCC
*** 220 called from CCCCCCCCCCCCCCCC STACK 000000000012FE60 AddrReturn CCCCCCCCCCCCCCCC
*** 221 called from CCCCCCCCCCCCCCCC STACK 000000000012FE68 AddrReturn CCCCCCCCCCCCCCCC
*** 222 called from CCCCCCCCCCCCCCCC STACK 000000000012FE70 AddrReturn 000000000012FEA0
*** 223 called from 000000000012FEA0 STACK 000000000012FE78 AddrReturn 000000014000128F
*** 224 called from 000000014000128F STACK 000000000012FE80 AddrReturn CCCCCCCCCCCCCCCC
*** 225 called from CCCCCCCCCCCCCCCC STACK 000000000012FE88 AddrReturn CCCCCCCCCCCCCCCC
*** 226 called from CCCCCCCCCCCCCCCC STACK 000000000012FE90 AddrReturn CCCCCCCCCCCCCCCC
*** 227 called from CCCCCCCCCCCCCCCC STACK 000000000012FE98 AddrReturn CCCCCCCCCCCCCCCC
*** 228 called from CCCCCCCCCCCCCCCC STACK 000000000012FEA0 AddrReturn 000000000012FED0
*** 229 called from 000000000012FED0 STACK 000000000012FEA8 AddrReturn 00000001400012CB
*** 230 called from 00000001400012CB STACK 000000000012FEB0 AddrReturn CCCCCCCCCCCCCCCC
*** 231 called from CCCCCCCCCCCCCCCC STACK 000000000012FEB8 AddrReturn CCCCCCCCCCCCCCCC
*** 232 called from CCCCCCCCCCCCCCCC STACK 000000000012FEC0 AddrReturn CCCCCCCCCCCCCCCC
*** 233 called from CCCCCCCCCCCCCCCC STACK 000000000012FEC8 AddrReturn CCCCCCCCCCCCCCCC
*** 234 called from CCCCCCCCCCCCCCCC STACK 000000000012FED0 AddrReturn 0000000000000000
*** 235 called from 0000000000000000 STACK 000000000012FED8 AddrReturn 000000014000190C
*** 236 called from 000000014000190C STACK 000000000012FEE0 AddrReturn 0000000100000001
*** 237 called from 0000000100000001 STACK 000000000012FEE8 AddrReturn 0000000000454B50
*** 238 called from 0000000000454B50 STACK 000000000012FEF0 AddrReturn 0000000000000000
*** 23
I finally found a reliable way to log the stack frames in x64, using the Windows function CaptureStackBackTrace(). As I did not want to update my SDK, I call it via GetProcAddress(LoadLibrary());
typedef USHORT (WINAPI *CaptureStackBackTraceType)(__in ULONG, __in ULONG, __out PVOID*, __out_opt PULONG);
CaptureStackBackTraceType func = (CaptureStackBackTraceType)(GetProcAddress(LoadLibrary("kernel32.dll"), "RtlCaptureStackBackTrace"));
if(func == NULL)
return; // WOE 29.SEP.2010
// Quote from Microsoft Documentation:
// ## Windows Server 2003 and Windows XP:
// ## The sum of the FramesToSkip and FramesToCapture parameters must be less than 63.
const int kMaxCallers = 62;
void* callers[kMaxCallers];
int count = (func)(0, kMaxCallers, callers, NULL);
for(i = 0; i < count; i++)
printf(TraceFile, "*** %d called from %016I64LX\n", i, callers[i]);
For vs2008 x64:
Based on https://msdn.microsoft.com/en-us/library/windows/desktop/bb204633%28v=vs.85%29.aspx and RED SOFT ADAIR:
#if defined DEBUG_SAMPLES_MANAGEMENT
#include "DbgHelp.h"
#include <WinBase.h>
#pragma comment(lib, "Dbghelp.lib")
void printStack( void* sample_address, std::fstream& out )
{
typedef USHORT (WINAPI *CaptureStackBackTraceType)(__in ULONG, __in ULONG, __out PVOID*, __out_opt PULONG);
CaptureStackBackTraceType func = (CaptureStackBackTraceType)(GetProcAddress(LoadLibrary(L"kernel32.dll"), "RtlCaptureStackBackTrace"));
if(func == NULL)
return; // WOE 29.SEP.2010
// Quote from Microsoft Documentation:
// ## Windows Server 2003 and Windows XP:
// ## The sum of the FramesToSkip and FramesToCapture parameters must be less than 63.
const int kMaxCallers = 62;
void * callers_stack[ kMaxCallers ];
unsigned short frames;
SYMBOL_INFO * symbol;
HANDLE process;
process = GetCurrentProcess();
SymInitialize( process, NULL, TRUE );
frames = (func)( 0, kMaxCallers, callers_stack, NULL );
symbol = ( SYMBOL_INFO * )calloc( sizeof( SYMBOL_INFO ) + 256 * sizeof( char ), 1 );
symbol->MaxNameLen = 255;
symbol->SizeOfStruct = sizeof( SYMBOL_INFO );
out << "(" << sample_address << "): " << std::endl;
const unsigned short MAX_CALLERS_SHOWN = 6;
frames = frames < MAX_CALLERS_SHOWN? frames : MAX_CALLERS_SHOWN;
for( unsigned int i = 0; i < frames; i++ )
{
SymFromAddr( process, ( DWORD64 )( callers_stack[ i ] ), 0, symbol );
out << "*** " << i << ": " << callers_stack[i] << " " << symbol->Name << " - 0x" << symbol->Address << std::endl;
}
free( symbol );
}
#endif
Called here:
#if defined DEBUG_SAMPLES_MANAGEMENT
if(owner_ != 0)
{
std::fstream& out = owner_->get_debug_file();
printStack( this, out );
}
#endif
We use minidumps exclusively here. You can generate a stripped down one that just includes stack information and dump out a stack trace from a decent debugger later.
It doesn't solve your problem directly, but I think it will provide you a much better postmortem reporting mechanism.
Disassembling RtlCaptureStackBackTrace() I noticed that maximum value passed to RtlCaptureStackBackTrace() should be: framesToSkip+framesToCapture+1 should be less than 64.
otherwise it returns 0 and there is no other error codes.
In Trial 3, you may be using CaptureStackBackTrace() incorrectly. According to the documentation, on Windows XP and Windows Server 2003, the sum of the first and second parameters must be less than 63, but, in your case, the sum would be 128.
http://msdn.microsoft.com/en-us/library/windows/desktop/bb204633%28v=vs.85%29.aspx
Watch this, I do not know if it is relevant:
...
Working with Assembly Code
Assembly code is straightforward to port to AMD64 and 64-bit Windows—and is worth the effort for performance reasons! For example, you can take advantage of the new 64-bit general-purpose registers (r8-r15), and new floating point and 128-bit SSE/SSE2/floating point registers (xmm8-xmm15). However, there are new 64-bit stack frames and calling conventions you should learn about in the ABI (application binary interface) specifications.
...
The trick is to stop calling StackWalk64 when it returns 0 in stk.AddrReturn.Offset. This means there are no more frames on the stack. If stk.AddrReturn.Offset is non-zero, you can use that value as the return address.
If you continue calling StackWalk64 after this, my guess is that it will try to interpret whatever is in the memory locations as being a stack and will return unpredictable data.
StackWalk64 is the right choice, the first call will give you the caller's address.
Your problem might be that in release you have a lot of inlining going on. The return address may not be what you expect.
edit : you only need to set AddrPC and AddrFrame. Just make sure that your rbp and rip are the ones corresponding to your callee context.
Regarding the first issue: disable "Omit stack frames" in thre release version, and the "trivial" stack tracing code will work.
Regarding RtlCaptureStackBackTrace, one thing I've noticed on 32-bit Windows is that it fails if you pass too large a number into it for FramesToCapture. Experimentally I've identified 61 as the maximum value, for no reason that I can fathom!
Not sure if it's the same in x64, but that might explain why you're getting no info out.
When using StackWalk64 you are iterating through the thread's entire stack whether there is valid data or not. Once you hit a return address of 0 you should terminate the walk, like so:
for (ULONG Frame = 0; ; Frame++)
{
if (FALSE == StackWalk64(...))
{
printf("Stack walk failed!\n");
break;
}
if (stackFrame.AddrPC.Offset == 0)
{
printf("Stack walk complete!\n");
break;
}
do_something();
}
Found the short version with the "CaptureStackBackTraceType" really useful!
Then resolved function names of the "callers[]" using
SymFromAddr(), SymInitialize(),
which is mainly the version from Pedro Reis
and finally demangled the function signatures according to
function to mangle/demangle functions
NOTE: GNU demangler function abi::__cxa_demangle() expects a single underscore prefix