Boost::stacktrace a specific thread - c++

I want to debug an application where some threads seem to go into a deadlock situation in a production environment (I cannot debug, thus I need a meaningful log). I found boost::stacktrace::stacktrace(), but it dumps info for all threads.
Is it possible to limit the output to a specific thread?

If it is Linux specific you can use backtrace and backtrace_symbols to retrieve the backtrace for that specific thread.
I adapted this example from man backtrace to include a thread.
#include <execinfo.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <thread>
#define BT_BUF_SIZE 100
void printbt(void)
{
int j, nptrs;
void *buffer[BT_BUF_SIZE];
char **strings;
nptrs = backtrace(buffer, BT_BUF_SIZE);
printf("backtrace() returned %d addresses\n", nptrs);
/* The call backtrace_symbols_fd(buffer, nptrs, STDOUT_FILENO)
would produce similar output to the following: */
strings = backtrace_symbols(buffer, nptrs);
if (strings == NULL) {
perror("backtrace_symbols");
exit(EXIT_FAILURE);
}
for (j = 0; j < nptrs; j++)
printf("%s\n", strings[j]);
free(strings);
}
static void myfunc2(void)
{
printbt();
}
void myfunc(int ncalls)
{
if (ncalls > 1)
myfunc(ncalls - 1);
else
myfunc2();
}
int main(int argc, char *argv[])
{
if (argc != 2) {
fprintf(stderr, "%s num-calls\n", argv[0]);
return 0;
}
std::thread th( myfunc, atoi(argv[1]) );
th.join();
}
You have to compile with -rdynamic to work
$ g++ -g3 -rdynamic test2.cpp -o /tmp/test2 -lpthread
Running it provides
$ /tmp/test2 3
backtrace() returned 13 addresses
/tmp/test2(_Z7printbtv+0x32) [0x55e2dbe5d37b]
/tmp/test2(+0x4451) [0x55e2dbe5d451]
/tmp/test2(_Z6myfunci+0x29) [0x55e2dbe5d47d]
/tmp/test2(_Z6myfunci+0x22) [0x55e2dbe5d476]
/tmp/test2(_Z6myfunci+0x22) [0x55e2dbe5d476]
/tmp/test2(_ZSt13__invoke_implIvPFviEJiEET_St14__invoke_otherOT0_DpOT1_+0x36) [0x55e2dbe5e0af]
/tmp/test2(_ZSt8__invokeIPFviEJiEENSt15__invoke_resultIT_JDpT0_EE4typeEOS3_DpOS4_+0x4a) [0x55e2dbe5dffd]
/tmp/test2(_ZNSt6thread8_InvokerISt5tupleIJPFviEiEEE9_M_invokeIJLm0ELm1EEEEvSt12_Index_tupleIJXspT_EEE+0x47) [0x55e2dbe5df4d]
/tmp/test2(_ZNSt6thread8_InvokerISt5tupleIJPFviEiEEEclEv+0x2b) [0x55e2dbe5deef]
/tmp/test2(_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJPFviEiEEEEE6_M_runEv+0x20) [0x55e2dbe5dec0]
/lib/x86_64-linux-gnu/libstdc++.so.6(+0xd6de4) [0x7f306312ede4]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f3063242609]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x43) [0x7f3062f6a133]
You can use __cxa_demangle to automatically demangle the symbols or you can simply use c++filt from gcc for that.
$ echo _ZSt13__invoke_implIvPFviEJiEET_St14__invoke_otherOT0_DpOT1_+0x36 | c++filt
void std::__invoke_impl<void, void (*)(int), int>(std::__invoke_other, void (*&&)(int), int&&)+0x36
$ echo _Z6myfunci+0x29 | c++filt
myfunc(int)+0x29
You can then go into gdb and look what line the offset 0x29 corresponds to inside the function myfunc(int)

Related

create thread but process shows up?

#include <unistd.h>
#include <stdio.h>
#include <cstring>
#include <thread>
void test_cpu() {
printf("thread: test_cpu start\n");
int total = 0;
while (1) {
++total;
}
}
void test_mem() {
printf("thread: test_mem start\n");
int step = 20;
int size = 10 * 1024 * 1024; // 10Mb
for (int i = 0; i < step; ++i) {
char* tmp = new char[size];
memset(tmp, i, size);
sleep(1);
}
printf("thread: test_mem done\n");
}
int main(int argc, char** argv) {
std::thread t1(test_cpu);
std::thread t2(test_mem);
t1.join();
t2.join();
return 0;
}
Compile it with g++ -o test test.cc --std=c++11 -lpthread
I run the program in Linux, and run top to monitor it.
I expect to see ONE process however I saw THREE.
It looks like std::thread is creating threads, why do I end up with getting processes?
Linux does not implement threads. It only has Light Weight Processes (LWP) while pthread library wraps them to provide POSIX-compatible thread interface. The main LWP creates its own address space while each subsequent thread LWP shares address space with main LWP.
Many utils, such as HTOP (which seems to be on the screenshot) by default list LWP. In order to hide thread LWPs you can open Setup (F2) -> Display Options and check Hide kernel threads and Hide userland process threads options. There is also an option to highlight threads - Display threads in different color.

Linux - segmentation fault only sometimes - how to debug

I have a Linux program, that from time to time ends with a segmentation fault. The program is running periodically every hour, but the segmentation fault occurs only sometimes.
I have a problem to debug this, because if I run the program again with the same input, no error is reported and all is OK.
Is there a way, how to "report" in which part of the code error occured or what caused the problem?
The usual way is to have the crashing program generate a corefile and analyze this after the crash. Make sure, that:
the maximum corefile-size is big enough (i.e. unlimited) by calling ulimit -c unlimited in the shell, which starts the process.
The cwd is writable by the segfaulting process.
Then you can analyze the file with
gdb <exe> <corefile>
Since your code not crashing every time, you can use backtrace as well. Using this you can see the function call stack at the time of crash. There are many examples available. In my projects I normally use the following code for backtracing.
/*
* call reg_usr2 function from main
* gcc -rdynamic myfile.c -o output
*/
#include <stdio.h>
#include <stdarg.h>
#include <signal.h>
#include <unistd.h>
#include <stdlib.h>
#include <execinfo.h>
#define FILE_NAME "/tmp/debug"
#define MODE 0xFFFF
void dbgprint(int flag, char* fmt, ...)
{
if(flag & MODE) {
char buf[100];
va_list vlist;
FILE *fp = fopen(FILE_NAME,"a");
va_start(vlist, fmt);
vsnprintf( buf, sizeof( buf), fmt, vlist);
va_end( vlist);
fprintf(fp,"[%x]->%s\n", flag, buf);
fclose(fp);
}
}
/** Here is the code to print backtrace **/
void print_stack_trace ()
{
void *array[20];
size_t size;
char **strings;
size_t i;
size = backtrace (array, 20);
strings = backtrace_symbols (array, size);
dbgprint(0xFFFF, "Obtained %zd stack frames.", size);
dbgprint(0xFFFF, "-------------------------");
dbgprint(0xFFFF, "---------Backtrace-------");
for (i = 0; i < size; i++)
dbgprint (0xFFFF, "%s", strings[i]);
dbgprint(0xFFFF, "-------------------------");
free (strings);
}
void sig_handler(int signo)
{
FILE *fp = fopen(FILE_NAME,"a");
if (signo == SIGUSR2){
dbgprint(0xFFFF, "received SIGUSR2");
dbgprint(0xFFFF, "----------------");
}
print_stack_trace();
exit(0);
}
void reg_usr2()
{
if (signal(SIGUSR2, sig_handler) == SIG_ERR)
printf("\ncan't catch SIGUSR2\n");
}
int main()
{
reg_usr2(); //should be first line of main after variables
//Code.....
return 0;
}
You can generate backtrace by catching SIGSEGV signal, and see where your application throw an invalid access.
see https://stackoverflow.com/a/77336/4490542
But there is more easier solution, try running your application with catchsegv
catchsegv './program args'
and better alternative, valgrind
valgrind --tool=none ./program args

Is it possible to set a gdb watchpoint programmatically?

I want to set a watchpoint (break on hardware write) temporarily in my C++ program to find memory corruption.
I've seen all the ways to do it manually through gdb, but I would like to actually set the watchpoint via some method in my code so I don't have to break into gdb, find out the address, set the watchpoint and then continue.
Something like:
#define SET_WATCHPOINT(addr) asm ("set break on hardware write %addr")
Set hardware watchpoint from child process.
#include <signal.h>
#include <syscall.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <stddef.h>
#include <sys/ptrace.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <linux/user.h>
enum {
DR7_BREAK_ON_EXEC = 0,
DR7_BREAK_ON_WRITE = 1,
DR7_BREAK_ON_RW = 3,
};
enum {
DR7_LEN_1 = 0,
DR7_LEN_2 = 1,
DR7_LEN_4 = 3,
};
typedef struct {
char l0:1;
char g0:1;
char l1:1;
char g1:1;
char l2:1;
char g2:1;
char l3:1;
char g3:1;
char le:1;
char ge:1;
char pad1:3;
char gd:1;
char pad2:2;
char rw0:2;
char len0:2;
char rw1:2;
char len1:2;
char rw2:2;
char len2:2;
char rw3:2;
char len3:2;
} dr7_t;
typedef void sighandler_t(int, siginfo_t*, void*);
int watchpoint(void* addr, sighandler_t handler)
{
pid_t child;
pid_t parent = getpid();
struct sigaction trap_action;
int child_stat = 0;
sigaction(SIGTRAP, NULL, &trap_action);
trap_action.sa_sigaction = handler;
trap_action.sa_flags = SA_SIGINFO | SA_RESTART | SA_NODEFER;
sigaction(SIGTRAP, &trap_action, NULL);
if ((child = fork()) == 0)
{
int retval = EXIT_SUCCESS;
dr7_t dr7 = {0};
dr7.l0 = 1;
dr7.rw0 = DR7_BREAK_ON_WRITE;
dr7.len0 = DR7_LEN_4;
if (ptrace(PTRACE_ATTACH, parent, NULL, NULL))
{
exit(EXIT_FAILURE);
}
sleep(1);
if (ptrace(PTRACE_POKEUSER, parent, offsetof(struct user, u_debugreg[0]), addr))
{
retval = EXIT_FAILURE;
}
if (ptrace(PTRACE_POKEUSER, parent, offsetof(struct user, u_debugreg[7]), dr7))
{
retval = EXIT_FAILURE;
}
if (ptrace(PTRACE_DETACH, parent, NULL, NULL))
{
retval = EXIT_FAILURE;
}
exit(retval);
}
waitpid(child, &child_stat, 0);
if (WEXITSTATUS(child_stat))
{
printf("child exit !0\n");
return 1;
}
return 0;
}
int var;
void trap(int sig, siginfo_t* info, void* context)
{
printf("new value: %d\n", var);
}
int main(int argc, char * argv[])
{
int i;
printf("init value: %d\n", var);
watchpoint(&var, trap);
for (i = 0; i < 100; i++) {
var++;
sleep(1);
}
return 0;
}
Based on user512106's great answer, I coded up a little "library" that someone might find useful:
It's on github at https://github.com/whh8b/hwbp_lib. I wish I could have commented directly on his answer, but I don't have enough rep yet.
Based on feedback from the community, I am going to copy/paste the relevant code here:
#include <stdio.h>
#include <stddef.h>
#include <signal.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <sys/ptrace.h>
#include <sys/user.h>
#include <sys/prctl.h>
#include <stdint.h>
#include <errno.h>
#include <stdbool.h>
extern int errno;
enum {
BREAK_EXEC = 0x0,
BREAK_WRITE = 0x1,
BREAK_READWRITE = 0x3,
};
enum {
BREAK_ONE = 0x0,
BREAK_TWO = 0x1,
BREAK_FOUR = 0x3,
BREAK_EIGHT = 0x2,
};
#define ENABLE_BREAKPOINT(x) (0x1<<(x*2))
#define ENABLE_BREAK_EXEC(x) (BREAK_EXEC<<(16+(x*4)))
#define ENABLE_BREAK_WRITE(x) (BREAK_WRITE<<(16+(x*4)))
#define ENABLE_BREAK_READWRITE(x) (BREAK_READWRITE<<(16+(x*4)))
/*
* This function fork()s a child that will use
* ptrace to set a hardware breakpoint for
* memory r/w at _addr_. When the breakpoint is
* hit, then _handler_ is invoked in a signal-
* handling context.
*/
bool install_breakpoint(void *addr, int bpno, void (*handler)(int)) {
pid_t child = 0;
uint32_t enable_breakpoint = ENABLE_BREAKPOINT(bpno);
uint32_t enable_breakwrite = ENABLE_BREAK_WRITE(bpno);
pid_t parent = getpid();
int child_status = 0;
if (!(child = fork()))
{
int parent_status = 0;
if (ptrace(PTRACE_ATTACH, parent, NULL, NULL))
_exit(1);
while (!WIFSTOPPED(parent_status))
waitpid(parent, &parent_status, 0);
/*
* set the breakpoint address.
*/
if (ptrace(PTRACE_POKEUSER,
parent,
offsetof(struct user, u_debugreg[bpno]),
addr))
_exit(1);
/*
* set parameters for when the breakpoint should be triggered.
*/
if (ptrace(PTRACE_POKEUSER,
parent,
offsetof(struct user, u_debugreg[7]),
enable_breakwrite | enable_breakpoint))
_exit(1);
if (ptrace(PTRACE_DETACH, parent, NULL, NULL))
_exit(1);
_exit(0);
}
waitpid(child, &child_status, 0);
signal(SIGTRAP, handler);
if (WIFEXITED(child_status) && !WEXITSTATUS(child_status))
return true;
return false;
}
/*
* This function will disable a breakpoint by
* invoking install_breakpoint is a 0x0 _addr_
* and no handler function. See comments above
* for implementation details.
*/
bool disable_breakpoint(int bpno)
{
return install_breakpoint(0x0, bpno, NULL);
}
/*
* Example of how to use this /library/.
*/
int handled = 0;
void handle(int s) {
handled = 1;
return;
}
int main(int argc, char **argv) {
int a = 0;
if (!install_breakpoint(&a, 0, handle))
printf("failed to set the breakpoint!\n");
a = 1;
printf("handled: %d\n", handled);
if (!disable_breakpoint(0))
printf("failed to disable the breakpoint!\n");
return 1;
}
I hope that this helps someone!
Will
In GDB, there are two types of watchpoints, hardware and software.
you can't implement easily software watchpoints: (cf. GDB Internals)
Software watchpoints are very slow, since gdb needs to single-step the program being debugged and test the value of the watched expression(s) after each instruction.
EDIT:
I'm still trying to understand what are hardware watchpoint.
for hardware breakpoints, this article gives some technics:
We want to watch reading from or writing into 1 qword at address 100005120h (address range 100005120h-100005127h)
lea rax, [100005120h]
mov dr0, rax
mov rax, dr7
and eax, not ((1111b shl 16) + 11b) ; mask off all
or eax, (1011b shl 16) + 1 ; prepare to set what we want
mov
dr7, rax ; set it finally
Done, now we can wait until code falls into the trap! After accessing any byte at memory range 100005120h-100005127h, int 1 will occur and DR6.B0 bit will be set to 1.
You can also take a look at GDB low-end files (eg, amd64-linux-nat.c) but it (certainly) involves 2 processes: 1/ the one you want to watch 2/a lightweight debugger who attaches to the first one with ptrace, and uses:
ptrace (PTRACE_POKEUSER, tid, __regnum__offset__, address);
to set and handle the watchpoint.
The program itself can supply commands to the GDB. You'll need a special shell script to run GDB though.
Copy this code into the file named untee, and execute chmod 755 untee
#!/bin/bash
if [ -z "$1" ]; then
echo "Usage: $0 PIPE | COMMAND"
echo "This script will read the input from both stdin and PIPE, and supply it to the COMMAND."
echo "If PIPE does not exist it will be created with mkfifo command."
exit 0
fi
PIPE="$1"
if [ \! -e "$PIPE" ]; then
mkfifo "$PIPE"
fi
if [ \! -p "$PIPE" ]; then
echo "File $PIPE does not exist or is not a named pipe" > /dev/stderr
exit 1
fi
# Open the pipe as a FD 3
echo "Waiting for $PIPE to be opened by another process" > /dev/stderr
exec 3<"$PIPE"
echo "$PIPE opened" > /dev/stderr
OPENED=true
while true; do
read -t 1 INPUT
RET=$?
if [ "$RET" = 0 ]; then
echo "$INPUT"
elif [ "$RET" -lt 128 ]; then
echo "stdin closed, exiting" > /dev/stderr
break
fi
if $OPENED; then
while read -t 1 -u 3 INPUT; do
RET=$?
if [ "$RET" = 0 ]; then
echo "$INPUT"
else
if [ "$RET" -lt 128 ]; then
echo "$PIPE closed, ignoring" > /dev/stderr
OPENED=false
fi
break
fi
done
fi
done
And now the C code:
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <signal.h>
#include <unistd.h>
void gdbCommand(const char *c)
{
static FILE * dbgpipe = NULL;
static const char * dbgpath = "/tmp/dbgpipe";
struct stat st;
if( !dbgpipe && stat(dbgpath, &st) == 0 && S_ISFIFO(st.st_mode) )
dbgpipe = fopen(dbgpath, "w");
if( !dbgpipe )
return;
fprintf(dbgpipe, "%s\n", c);
fflush(dbgpipe);
}
void gdbSetWatchpoint(const char *var)
{
char buf[256];
snprintf(buf, sizeof(buf), "watch %s", var);
gdbCommand("up"); /* Go up the stack from the kill() system call - this may vary by the OS, you may need to walk the stack more times */
gdbCommand("up"); /* Go up the stack from the gdbSetWatchpoint() function */
gdbCommand(buf);
gdbCommand("continue");
kill(getpid(), SIGINT); /* Make GDB pause our process and execute commands */
}
int subfunc(int *v)
{
*v += 5; /* GDB should pause after this line, and let you explore stack etc */
return v;
}
int func()
{
int i = 10;
printf("Adding GDB watch for var 'i'\n");
gdbSetWatchpoint("i");
subfunc(&i);
return i;
}
int func2()
{
int j = 20;
return j + func();
}
int main(int argc, char ** argv)
{
func();
func2();
return 0;
}
Copy that to the file named test.c, compile with command gcc test.c -O0 -g -o test then execute ./untee /tmp/dbgpipe | gdb -ex "run" ./test
This works on my 64-bit Ubuntu, with GDB 7.3 (older GDB versions might refuse to read commands from non-terminal)
If you happen to be using Xcode, you can achieve the required effect (automatic setting of watchpoints) by using an action on another breakpoint to set your watchpoint:
Set up a breakpoint somewhere where the variable you want to watch will be in scope that will be hit before you need to start watching the variable,
Right-click on the breakpoint and select Edit Breakpoint...,
Click on Add Action and add a Debugger Command with an LLDB command like: watchpoint set variable <variablename> (or if you're using GDB1, a command like: watch <variablename>),
Check the Automatically continue after evaluating actions checkbox.
1: GDB is no longer supported in more recent versions of Xcode, but I believe it is still possible to set it up manually.

How to get a stack trace for C++ using gcc with line number information?

We use stack traces in proprietary assert like macro to catch developer mistakes - when error is caught, stack trace is printed.
I find gcc's pair backtrace()/backtrace_symbols() methods insufficient:
Names are mangled
No line information
1st problem can be resolved by abi::__cxa_demangle.
However 2nd problem s more tough. I found replacement for backtrace_symbols().
This is better than gcc's backtrace_symbols(), since it can retrieve line numbers (if compiled with -g) and you don't need to compile with -rdynamic.
Hoverer the code is GNU licenced, so IMHO I can't use it in commercial code.
Any proposal?
P.S.
gdb is capable to print out arguments passed to functions.
Probably it's already too much to ask for :)
PS 2
Similar question (thanks nobar)
So you want a stand-alone function that prints a stack trace with all of the features that gdb stack traces have and that doesn't terminate your application. The answer is to automate the launch of gdb in a non-interactive mode to perform just the tasks that you want.
This is done by executing gdb in a child process, using fork(), and scripting it to display a stack-trace while your application waits for it to complete. This can be performed without the use of a core-dump and without aborting the application. I learned how to do this from looking at this question: How it's better to invoke gdb from program to print it's stacktrace?
The example posted with that question didn't work for me exactly as written, so here's my "fixed" version (I ran this on Ubuntu 9.04).
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
#include <sys/prctl.h>
void print_trace() {
char pid_buf[30];
sprintf(pid_buf, "%d", getpid());
char name_buf[512];
name_buf[readlink("/proc/self/exe", name_buf, 511)]=0;
prctl(PR_SET_PTRACER, PR_SET_PTRACER_ANY, 0, 0, 0);
int child_pid = fork();
if (!child_pid) {
dup2(2,1); // redirect output to stderr - edit: unnecessary?
execl("/usr/bin/gdb", "gdb", "--batch", "-n", "-ex", "thread", "-ex", "bt", name_buf, pid_buf, NULL);
abort(); /* If gdb failed to start */
} else {
waitpid(child_pid,NULL,0);
}
}
As shown in the referenced question, gdb provides additional options that you could use. For example, using "bt full" instead of "bt" produces an even more detailed report (local variables are included in the output). The manpages for gdb are kind of light, but complete documentation is available here.
Since this is based on gdb, the output includes demangled names, line-numbers, function arguments, and optionally even local variables. Also, gdb is thread-aware, so you should be able to extract some thread-specific metadata.
Here's an example of the kind of stack traces that I see with this method.
0x00007f97e1fc2925 in waitpid () from /lib/libc.so.6
[Current thread is 0 (process 15573)]
#0 0x00007f97e1fc2925 in waitpid () from /lib/libc.so.6
#1 0x0000000000400bd5 in print_trace () at ./demo3b.cpp:496
2 0x0000000000400c09 in recursive (i=2) at ./demo3b.cpp:636
3 0x0000000000400c1a in recursive (i=1) at ./demo3b.cpp:646
4 0x0000000000400c1a in recursive (i=0) at ./demo3b.cpp:646
5 0x0000000000400c46 in main (argc=1, argv=0x7fffe3b2b5b8) at ./demo3b.cpp:70
Note: I found this to be incompatible with the use of valgrind (probably due to Valgrind's use of a virtual machine). It also doesn't work when you are running the program inside of a gdb session (can't apply a second instance of "ptrace" to a process).
Not too long ago I answered a similar question. You should take a look at the source code available on method #4, which also prints line numbers and filenames.
Method #4:
A small improvement I've done on method #3 to print line numbers. This could be copied to work on method #2 also.
Basically, it uses addr2line to convert addresses into file names and line numbers.
The source code below prints line numbers for all local functions. If a function from another library is called, you might see a couple of ??:0 instead of file names.
#include <stdio.h>
#include <signal.h>
#include <stdio.h>
#include <signal.h>
#include <execinfo.h>
void bt_sighandler(int sig, struct sigcontext ctx) {
void *trace[16];
char **messages = (char **)NULL;
int i, trace_size = 0;
if (sig == SIGSEGV)
printf("Got signal %d, faulty address is %p, "
"from %p\n", sig, ctx.cr2, ctx.eip);
else
printf("Got signal %d\n", sig);
trace_size = backtrace(trace, 16);
/* overwrite sigaction with caller's address */
trace[1] = (void *)ctx.eip;
messages = backtrace_symbols(trace, trace_size);
/* skip first stack frame (points here) */
printf("[bt] Execution path:\n");
for (i=1; i<trace_size; ++i)
{
printf("[bt] #%d %s\n", i, messages[i]);
/* find first occurence of '(' or ' ' in message[i] and assume
* everything before that is the file name. (Don't go beyond 0 though
* (string terminator)*/
size_t p = 0;
while(messages[i][p] != '(' && messages[i][p] != ' '
&& messages[i][p] != 0)
++p;
char syscom[256];
sprintf(syscom,"addr2line %p -e %.*s", trace[i], p, messages[i]);
//last parameter is the file name of the symbol
system(syscom);
}
exit(0);
}
int func_a(int a, char b) {
char *p = (char *)0xdeadbeef;
a = a + b;
*p = 10; /* CRASH here!! */
return 2*a;
}
int func_b() {
int res, a = 5;
res = 5 + func_a(a, 't');
return res;
}
int main() {
/* Install our signal handler */
struct sigaction sa;
sa.sa_handler = (void *)bt_sighandler;
sigemptyset(&sa.sa_mask);
sa.sa_flags = SA_RESTART;
sigaction(SIGSEGV, &sa, NULL);
sigaction(SIGUSR1, &sa, NULL);
/* ... add any other signal here */
/* Do something */
printf("%d\n", func_b());
}
This code should be compiled as: gcc sighandler.c -o sighandler -rdynamic
The program outputs:
Got signal 11, faulty address is 0xdeadbeef, from 0x8048975
[bt] Execution path:
[bt] #1 ./sighandler(func_a+0x1d) [0x8048975]
/home/karl/workspace/stacktrace/sighandler.c:44
[bt] #2 ./sighandler(func_b+0x20) [0x804899f]
/home/karl/workspace/stacktrace/sighandler.c:54
[bt] #3 ./sighandler(main+0x6c) [0x8048a16]
/home/karl/workspace/stacktrace/sighandler.c:74
[bt] #4 /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0x3fdbd6]
??:0
[bt] #5 ./sighandler() [0x8048781]
??:0
There is a robust discussion of essentially the same question at: How to generate a stacktrace when my gcc C++ app crashes. Many suggestions are provided, including lots of discussion about how to generate stack traces at run-time.
My personal favorite answer from that thread was to enable core dumps which allows you to view the complete application state at the time of the crash (including function arguments, line numbers, and unmangled names). An additional benefit of this approach is that it not only works for asserts, but also for segmentation faults and unhandled exceptions.
Different Linux shells use different commands to enable core dumps, but you can do it from within your application code with something like this...
#include <sys/resource.h>
...
struct rlimit core_limit = { RLIM_INFINITY, RLIM_INFINITY };
assert( setrlimit( RLIMIT_CORE, &core_limit ) == 0 ); // enable core dumps for debug builds
After a crash, run your favorite debugger to examine the program state.
$ kdbg executable core
Here's some sample output...
It is also possible to extract the stack trace from a core dump at the command line.
$ ( CMDFILE=$(mktemp); echo "bt" >${CMDFILE}; gdb 2>/dev/null --batch -x ${CMDFILE} temp.exe core )
Core was generated by `./temp.exe'.
Program terminated with signal 6, Aborted.
[New process 22857]
#0 0x00007f4189be5fb5 in raise () from /lib/libc.so.6
#0 0x00007f4189be5fb5 in raise () from /lib/libc.so.6
#1 0x00007f4189be7bc3 in abort () from /lib/libc.so.6
#2 0x00007f4189bdef09 in __assert_fail () from /lib/libc.so.6
#3 0x00000000004007e8 in recursive (i=5) at ./demo1.cpp:18
#4 0x00000000004007f3 in recursive (i=4) at ./demo1.cpp:19
#5 0x00000000004007f3 in recursive (i=3) at ./demo1.cpp:19
#6 0x00000000004007f3 in recursive (i=2) at ./demo1.cpp:19
#7 0x00000000004007f3 in recursive (i=1) at ./demo1.cpp:19
#8 0x00000000004007f3 in recursive (i=0) at ./demo1.cpp:19
#9 0x0000000000400849 in main (argc=1, argv=0x7fff2483bd98) at ./demo1.cpp:26
Since the GPL licensed code is intended to help you during development, you could simply not include it in the final product. The GPL restricts you from distributing GPL licenses code linked with non-GPL compatible code. As long as you only use the GPL code inhouse, you should be fine.
Use the google glog library for it. It has new BSD licence.
It contains a GetStackTrace function in the stacktrace.h file.
EDIT
I found here http://blog.bigpixel.ro/2010/09/09/stack-unwinding-stack-trace-with-gcc/ that there is an utility called addr2line that translates program addresses into file names and line numbers.
http://linuxcommand.org/man_pages/addr2line1.html
Here's an alternative approach. A debug_assert() macro programmatically sets a conditional breakpoint. If you are running in a debugger, you will hit a breakpoint when the assert expression is false -- and you can analyze the live stack (the program doesn't terminate). If you are not running in a debugger, a failed debug_assert() causes the program to abort and you get a core dump from which you can analyze the stack (see my earlier answer).
The advantage of this approach, compared to normal asserts, is that you can continue running the program after the debug_assert is triggered (when running in a debugger). In other words, debug_assert() is slightly more flexible than assert().
#include <iostream>
#include <cassert>
#include <sys/resource.h>
// note: The assert expression should show up in
// stack trace as parameter to this function
void debug_breakpoint( char const * expression )
{
asm("int3"); // x86 specific
}
#ifdef NDEBUG
#define debug_assert( expression )
#else
// creates a conditional breakpoint
#define debug_assert( expression ) \
do { if ( !(expression) ) debug_breakpoint( #expression ); } while (0)
#endif
void recursive( int i=0 )
{
debug_assert( i < 5 );
if ( i < 10 ) recursive(i+1);
}
int main( int argc, char * argv[] )
{
rlimit core_limit = { RLIM_INFINITY, RLIM_INFINITY };
setrlimit( RLIMIT_CORE, &core_limit ); // enable core dumps
recursive();
}
Note: Sometimes "conditional breakpoints" setup within debuggers can be slow. By establishing the breakpoint programmatically, the performance of this method should be equivalent to that of a normal assert().
Note: As written, this is specific to the Intel x86 architecture -- other processors may have different instructions for generating a breakpoint.
A bit late, but you can use libbfb to fetch the filename and linenumber like refdbg does in symsnarf.c. libbfb is internally used by addr2line and gdb
here is my solution:
#include <execinfo.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <stdlib.h>
#include <iostream>
#include <zconf.h>
#include "regex"
std::string getexepath() {
char result[PATH_MAX];
ssize_t count = readlink("/proc/self/exe", result, PATH_MAX);
return std::string(result, (count > 0) ? count : 0);
}
std::string sh(std::string cmd) {
std::array<char, 128> buffer;
std::string result;
std::shared_ptr<FILE> pipe(popen(cmd.c_str(), "r"), pclose);
if (!pipe) throw std::runtime_error("popen() failed!");
while (!feof(pipe.get())) {
if (fgets(buffer.data(), 128, pipe.get()) != nullptr) {
result += buffer.data();
}
}
return result;
}
void print_backtrace(void) {
void *bt[1024];
int bt_size;
char **bt_syms;
int i;
bt_size = backtrace(bt, 1024);
bt_syms = backtrace_symbols(bt, bt_size);
std::regex re("\\[(.+)\\]");
auto exec_path = getexepath();
for (i = 1; i < bt_size; i++) {
std::string sym = bt_syms[i];
std::smatch ms;
if (std::regex_search(sym, ms, re)) {
std::string addr = ms[1];
std::string cmd = "addr2line -e " + exec_path + " -f -C " + addr;
auto r = sh(cmd);
std::regex re2("\\n$");
auto r2 = std::regex_replace(r, re2, "");
std::cout << r2 << std::endl;
}
}
free(bt_syms);
}
void test_m() {
print_backtrace();
}
int main() {
test_m();
return 0;
}
output:
/home/roroco/Dropbox/c/ro-c/cmake-build-debug/ex/test_backtrace_with_line_number
test_m()
/home/roroco/Dropbox/c/ro-c/ex/test_backtrace_with_line_number.cpp:57
main
/home/roroco/Dropbox/c/ro-c/ex/test_backtrace_with_line_number.cpp:61
??
??:0
"??" and "??:0" since this trace is in libc, not in my source
The one of solutions is to start a gdb with "bt"-script in failed assert handler. It is not very easy to integrate such gdb-starting, but It will give you both backtrace and args and demangle names (or you can pass gdb output via c++filt programm).
Both programms (gdb and c++filt) will be not linked into your application, so GPL will not require you to opensource complete application.
The same approach (exec a GPL programme) you can use with backtrace-symbols. Just generate ascii list of %eip's and map of exec file (/proc/self/maps) and pass it to separate binary.
You can use DeathHandler - small C++ class which does everything for you, reliable.
I suppose line numbers are related to current eip value, right?
SOLUTION 1:
Then you can use something like GetThreadContext(), except that you're working on linux. I googled around a bit and found something similar, ptrace():
The ptrace() system call provides a
means by which a parent process may
observe and control the execution of
another process, and examine and
change its core image and registers. [...]
The parent can initiate a trace by
calling fork(2) and having the
resulting child do a PTRACE_TRACEME,
followed (typically) by an exec(3).
Alternatively, the parent may commence
trace of an existing process using
PTRACE_ATTACH.
Now I was thinking, you can do a 'main' program which checks for signals that are sent to its child, the real program you're working on. after fork() it call waitid():
All of these system calls are used to
wait for state changes in a child of
the calling process, and obtain
information about the child whose
state has changed.
and if a SIGSEGV (or something similar) is caught call ptrace() to obtain eip's value.
PS: I've never used these system calls (well, actually, I've never seen them before ;) so I don't know if it's possible neither can help you. At least I hope these links are useful. ;)
SOLUTION 2:
The first solution is quite complicated, right? I came up with a much simpler one: using signal() catch the signals you are interested in and call a simple function that reads the eip value stored in the stack:
...
signal(SIGSEGV, sig_handler);
...
void sig_handler(int signum)
{
int eip_value;
asm {
push eax;
mov eax, [ebp - 4]
mov eip_value, eax
pop eax
}
// now you have the address of the
// **next** instruction after the
// SIGSEGV was received
}
That asm syntax is Borland's one, just adapt it to GAS. ;)
Here's my third answer -- still trying to take advantage of core dumps.
It wasn't completely clear in the question whether the "assert-like" macros were supposed to terminate the application (the way assert does) or they were supposed to continue executing after generating their stack-trace.
In this answer, I'm addressing the case where you want to show a stack-trace and continue executing. I wrote the coredump() function below to generate a core dump, automatically extract the stack-trace from it, then continue executing the program.
Usage is the same as that of assert(). The difference, of course, is that assert() terminates the program but coredump_assert() does not.
#include <iostream>
#include <sys/resource.h>
#include <cstdio>
#include <cstdlib>
#include <boost/lexical_cast.hpp>
#include <string>
#include <sys/wait.h>
#include <unistd.h>
std::string exename;
// expression argument is for diagnostic purposes (shows up in call-stack)
void coredump( char const * expression )
{
pid_t childpid = fork();
if ( childpid == 0 ) // child process generates core dump
{
rlimit core_limit = { RLIM_INFINITY, RLIM_INFINITY };
setrlimit( RLIMIT_CORE, &core_limit ); // enable core dumps
abort(); // terminate child process and generate core dump
}
// give each core-file a unique name
if ( childpid > 0 ) waitpid( childpid, 0, 0 );
static int count=0;
using std::string;
string pid = boost::lexical_cast<string>(getpid());
string newcorename = "core-"+boost::lexical_cast<string>(count++)+"."+pid;
string rawcorename = "core."+boost::lexical_cast<string>(childpid);
int rename_rval = rename(rawcorename.c_str(),newcorename.c_str()); // try with core.PID
if ( rename_rval == -1 ) rename_rval = rename("core",newcorename.c_str()); // try with just core
if ( rename_rval == -1 ) std::cerr<<"failed to capture core file\n";
#if 1 // optional: dump stack trace and delete core file
string cmd = "( CMDFILE=$(mktemp); echo 'bt' >${CMDFILE}; gdb 2>/dev/null --batch -x ${CMDFILE} "+exename+" "+newcorename+" ; unlink ${CMDFILE} )";
int system_rval = system( ("bash -c '"+cmd+"'").c_str() );
if ( system_rval == -1 ) std::cerr.flush(), perror("system() failed during stack trace"), fflush(stderr);
unlink( newcorename.c_str() );
#endif
}
#ifdef NDEBUG
#define coredump_assert( expression ) ((void)(expression))
#else
#define coredump_assert( expression ) do { if ( !(expression) ) { coredump( #expression ); } } while (0)
#endif
void recursive( int i=0 )
{
coredump_assert( i < 2 );
if ( i < 4 ) recursive(i+1);
}
int main( int argc, char * argv[] )
{
exename = argv[0]; // this is used to generate the stack trace
recursive();
}
When I run the program, it displays three stack traces...
Core was generated by `./temp.exe'.
Program terminated with signal 6, Aborted.
[New process 24251]
#0 0x00007f2818ac9fb5 in raise () from /lib/libc.so.6
#0 0x00007f2818ac9fb5 in raise () from /lib/libc.so.6
#1 0x00007f2818acbbc3 in abort () from /lib/libc.so.6
#2 0x0000000000401a0e in coredump (expression=0x403303 "i < 2") at ./demo3.cpp:29
#3 0x0000000000401f5f in recursive (i=2) at ./demo3.cpp:60
#4 0x0000000000401f70 in recursive (i=1) at ./demo3.cpp:61
#5 0x0000000000401f70 in recursive (i=0) at ./demo3.cpp:61
#6 0x0000000000401f8b in main (argc=1, argv=0x7fffc229eb98) at ./demo3.cpp:66
Core was generated by `./temp.exe'.
Program terminated with signal 6, Aborted.
[New process 24259]
#0 0x00007f2818ac9fb5 in raise () from /lib/libc.so.6
#0 0x00007f2818ac9fb5 in raise () from /lib/libc.so.6
#1 0x00007f2818acbbc3 in abort () from /lib/libc.so.6
#2 0x0000000000401a0e in coredump (expression=0x403303 "i < 2") at ./demo3.cpp:29
#3 0x0000000000401f5f in recursive (i=3) at ./demo3.cpp:60
#4 0x0000000000401f70 in recursive (i=2) at ./demo3.cpp:61
#5 0x0000000000401f70 in recursive (i=1) at ./demo3.cpp:61
#6 0x0000000000401f70 in recursive (i=0) at ./demo3.cpp:61
#7 0x0000000000401f8b in main (argc=1, argv=0x7fffc229eb98) at ./demo3.cpp:66
Core was generated by `./temp.exe'.
Program terminated with signal 6, Aborted.
[New process 24267]
#0 0x00007f2818ac9fb5 in raise () from /lib/libc.so.6
#0 0x00007f2818ac9fb5 in raise () from /lib/libc.so.6
#1 0x00007f2818acbbc3 in abort () from /lib/libc.so.6
#2 0x0000000000401a0e in coredump (expression=0x403303 "i < 2") at ./demo3.cpp:29
#3 0x0000000000401f5f in recursive (i=4) at ./demo3.cpp:60
#4 0x0000000000401f70 in recursive (i=3) at ./demo3.cpp:61
#5 0x0000000000401f70 in recursive (i=2) at ./demo3.cpp:61
#6 0x0000000000401f70 in recursive (i=1) at ./demo3.cpp:61
#7 0x0000000000401f70 in recursive (i=0) at ./demo3.cpp:61
#8 0x0000000000401f8b in main (argc=1, argv=0x7fffc229eb98) at ./demo3.cpp:66
I had to do this in a production environment with many constraints, so I wanted to explain the advantages and disadvantages of the already posted methods.
attach GDB
+ very simple and robust
- Slow for large programs because GDB insists on loading the entire address to line # database upfront instead of lazily
- Interferes with signal handling. When GDB is attached, it intercepts signals like SIGINT (ctrl-c), which will cause the program to get stuck at the GDB interactive prompt? if some other process routinely sends such signals. Maybe there's some way around it, but this made GDB unusable in my case. You can still use it if you only care about printing a call stack once when your program crashes, but not multiple times.
addr2line. Here's an alternate solution that doesn't use backtrace_symbols.
+ Doesn't allocate from the heap, which is unsafe inside a signal handler
+ Don't need to parse output of backtrace_symbols
- Won't work on MacOS, which doesn't have dladdr1. You can use _dyld_get_image_vmaddr_slide instead, which returns the same offset as link_map::l_addr.
- Requires adding negative offset or else the translated line # will be 1 greater. backtrace_symbols does this for you
#include <execinfo.h>
#include <link.h>
#include <stdlib.h>
#include <stdio.h>
// converts a function's address in memory to its VMA address in the executable file. VMA is what addr2line expects
size_t ConvertToVMA(size_t addr)
{
Dl_info info;
link_map* link_map;
dladdr1((void*)addr,&info,(void**)&link_map,RTLD_DL_LINKMAP);
return addr-link_map->l_addr;
}
void PrintCallStack()
{
void *callstack[128];
int frame_count = backtrace(callstack, sizeof(callstack)/sizeof(callstack[0]));
for (int i = 0; i < frame_count; i++)
{
char location[1024];
Dl_info info;
if(dladdr(callstack[i],&info))
{
char command[256];
size_t VMA_addr=ConvertToVMA((size_t)callstack[i]);
//if(i!=crash_depth)
VMA_addr-=1; // https://stackoverflow.com/questions/11579509/wrong-line-numbers-from-addr2line/63841497#63841497
snprintf(command,sizeof(command),"addr2line -e %s -Ci %zx",info.dli_fname,VMA_addr);
system(command);
}
}
}
void Foo()
{
PrintCallStack();
}
int main()
{
Foo();
return 0;
}
I also want to clarify what addresses backtrace and backtrace_symbols generate and what addr2line expects.
addr2line expects FooVMA or if you're using --section=.text, then Foofile - textfile. backtrace returns Foomem. backtrace_symbols generates FooVMA somewhere.
One big mistake I made and saw in several other posts was assuming VMAbase = 0 or FooVMA = Foofile = Foomem - ELFmem, which is easy to calculate.
That often works, but for some compilers (i.e. linker scripts) use VMAbase > 0. Examples would be the GCC 5.4 on Ubuntu 16 (0x400000) and clang 11 on MacOS (0x100000000).
For shared libs, it's always 0. Seems VMAbase was only meaningful for non-position independent code. Otherwise it has no effect on where the EXE is loaded in memory.
Also, neither karlphillip's nor this one requires compiling with -rdynamic. That will increase the binary size, especially for a large C++ program or shared lib, with useless entries in the dynamic symbol table that never get imported
AFAICS all of the solutions provided so far won't print functions names and line numbers from shared libraries. That's what I needed, so i altered karlphillip's solution (and some other answer from a similar question) to resolve shared library addresses using /proc/id/maps.
#include <stdlib.h>
#include <inttypes.h>
#include <stdio.h>
#include <string.h>
#include <execinfo.h>
#include <stdbool.h>
struct Region { // one mapped file, for example a shared library
uintptr_t start;
uintptr_t end;
char* path;
};
static struct Region* getRegions(int* size) {
// parse /proc/self/maps and get list of mapped files
FILE* file;
int allocated = 10;
*size = 0;
struct Region* res;
uintptr_t regionStart = 0x00000000;
uintptr_t regionEnd = 0x00000000;
char* regionPath = "";
uintmax_t matchedStart;
uintmax_t matchedEnd;
char* matchedPath;
res = (struct Region*)malloc(sizeof(struct Region) * allocated);
file = fopen("/proc/self/maps", "r");
while (!feof(file)) {
fscanf(file, "%jx-%jx %*s %*s %*s %*s%*[ ]%m[^\n]\n", &matchedStart, &matchedEnd, &matchedPath);
bool bothNull = matchedPath == 0x0 && regionPath == 0x0;
bool similar = matchedPath && regionPath && !strcmp(matchedPath, regionPath);
if(bothNull || similar) {
free(matchedPath);
regionEnd = matchedEnd;
} else {
if(*size == allocated) {
allocated *= 2;
res = (struct Region*)realloc(res, sizeof(struct Region) * allocated);
}
res[*size].start = regionStart;
res[*size].end = regionEnd;
res[*size].path = regionPath;
(*size)++;
regionStart = matchedStart;
regionEnd = matchedEnd;
regionPath = matchedPath;
}
}
return res;
}
struct SemiResolvedAddress {
char* path;
uintptr_t offset;
};
static struct SemiResolvedAddress semiResolve(struct Region* regions, int regionsNum, uintptr_t address) {
// convert address from our address space to
// address suitable fo addr2line
struct Region* region;
struct SemiResolvedAddress res = {"", address};
for(region = regions; region < regions+regionsNum; region++) {
if(address >= region->start && address < region->end) {
res.path = region->path;
res.offset = address - region->start;
}
}
return res;
}
void printStacktraceWithLines(unsigned int max_frames)
{
int regionsNum;
fprintf(stderr, "stack trace:\n");
// storage array for stack trace address data
void* addrlist[max_frames+1];
// retrieve current stack addresses
int addrlen = backtrace(addrlist, sizeof(addrlist) / sizeof(void*));
if (addrlen == 0) {
fprintf(stderr, " <empty, possibly corrupt>\n");
return;
}
struct Region* regions = getRegions(&regionsNum);
for (int i = 1; i < addrlen; i++)
{
struct SemiResolvedAddress hres =
semiResolve(regions, regionsNum, (uintptr_t)(addrlist[i]));
char syscom[256];
sprintf(syscom, "addr2line -C -f -p -a -e %s 0x%jx", hres.path, (intmax_t)(hres.offset));
system(syscom);
}
free(regions);
}
C++23 <stacktrace>
Finally, this has arrived! More details/comparison with other systems at: print call stack in C or C++
stacktrace.cpp
#include <iostream>
#include <stacktrace>
void my_func_2(void) {
std::cout << std::stacktrace::current(); // Line 5
}
void my_func_1(double f) {
(void)f;
my_func_2(); // Line 10
}
void my_func_1(int i) {
(void)i;
my_func_2(); // Line 15
}
int main(int argc, char **argv) {
my_func_1(1); // Line 19
my_func_1(2.0); // Line 20
}
GCC 12.1.0 from Ubuntu 22.04 does not have support compiled in, so for now I built it from source as per: How to edit and re-build the GCC libstdc++ C++ standard library source? and set --enable-libstdcxx-backtrace=yes, and it worked!
Compile and run:
g++ -ggdb3 -O2 -std=c++23 -Wall -Wextra -pedantic -o stacktrace.out stacktrace.cpp -lstdc++_libbacktrace
./stacktrace.out
Output:
0# my_func_2() at /home/ciro/stacktrace.cpp:5
1# my_func_1(int) at /home/ciro/stacktrace.cpp:15
2# at :0
3# at :0
4# at :0
5#
0# my_func_2() at /home/ciro/stacktrace.cpp:5
1# my_func_1(double) at /home/ciro/stacktrace.cpp:10
2# at :0
3# at :0
4# at :0
5#
The trace is not perfect (missing main line) because of optimization I think. With -O0 it is better:
0# my_func_2() at /home/ciro/stacktrace.cpp:5
1# my_func_1(int) at /home/ciro/stacktrace.cpp:15
2# at /home/ciro/stacktrace.cpp:19
3# at :0
4# at :0
5# at :0
6#
0# my_func_2() at /home/ciro/stacktrace.cpp:5
1# my_func_1(double) at /home/ciro/stacktrace.cpp:10
2# at /home/ciro/stacktrace.cpp:20
3# at :0
4# at :0
5# at :0
6#
I don't know why the name main is missing, but the line is there.
The "extra" lines after main like:
3# at :0
4# at :0
5# at :0
6#
are probably stuff that runs before main and that ends up calling main: What happens before main in C++?

print call stack in C or C++

Is there any way to dump the call stack in a running process in C or C++ every time a certain function is called? What I have in mind is something like this:
void foo()
{
print_stack_trace();
// foo's body
return
}
Where print_stack_trace works similarly to caller in Perl.
Or something like this:
int main (void)
{
// will print out debug info every time foo() is called
register_stack_trace_function(foo);
// etc...
}
where register_stack_trace_function puts some sort of internal breakpoint that will cause a stack trace to be printed whenever foo is called.
Does anything like this exist in some standard C library?
I am working on Linux, using GCC.
Background
I have a test run that behaves differently based on some commandline switches that shouldn't affect this behavior. My code has a pseudo-random number generator that I assume is being called differently based on these switches. I want to be able to run the test with each set of switches and see if the random number generator is called differently for each one.
Survey of C/C++ backtrace methods
In this answer I will try to run a single benchmark for a bunch of solutions to see which one runs faster, while also considering other points such as features and portability.
Tool
Time / call
Line number
Function name
C++ demangling
Recompile
Signal safe
As string
C
C++23 <stacktrace> GCC 12.1
7 us
y
y
y
y
n
y
n
Boost 1.74 stacktrace()
5 us
y
y
y
y
n
y
n
Boost 1.74 stacktrace::safe_dump_to
y
n
n
glibc backtrace_symbols_fd
25 us
n
-rdynamic
hacks
y
y
n
y
glibc backtrace_symbols
21 us
n
-rdynamic
hacks
y
n
y
y
GDB scripting
600 us
y
y
y
n
y
n
y
GDB code injection
n
n
y
libunwind
y
libdwfl
4 ms
n
y
libbacktrace
y
Empty cells mean "TODO", not "no".
us: microsecond
Line number: shows actual line number, not just function name + a memory address.
It is usually possible to recover the line number from an address manually after the fact with addr2line. But it is a pain.
Recompile: requires recompiling the program to get your traces. Not recompiling is better!
Signal safe: crucial for the important uses case of "getting a stack trace in case of segfault": How to automatically generate a stacktrace when my program crashes
As string: you get the stack trace as a string in the program itself, as opposed to e.g. just printing to stdout. Usually implies not signal safe, as we don't know the size of the stack trace string size in advance, and therefore requires malloc which is not async signal safe.
C: does it work on a plain-C project (yes, there are still poor souls out there), or is C++ required?
Test setup
All benchmarks will run the following
main.cpp
#include <cstdlib> // strtoul
#include <mystacktrace.h>
void my_func_2(void) {
print_stacktrace(); // line 6
}
void my_func_1(double f) {
(void)f;
my_func_2();
}
void my_func_1(int i) {
(void)i;
my_func_2(); // line 16
}
int main(int argc, char **argv) {
long long unsigned int n;
if (argc > 1) {
n = std::strtoul(argv[1], NULL, 0);
} else {
n = 1;
}
for (long long unsigned int i = 0; i < n; ++i) {
my_func_1(1); // line 27
}
}
This input is designed to test C++ name demangling since my_func_1(int) and my_func_1(float) are necessarily mangled as a way to implement C++ function overload.
We differentiate between the benchmarks by using different -I includes to point to different implementations of print_stacktrace().
Each benchmark is done with a command of form:
time ./stacktrace.out 100000 &>/dev/null
The number of iterations is adjusted for each implementation to produce a total runtime of the order of 1s for that benchmark.
-O0 is used on all tests below unless noted. Stack traces may be irreparably mutilated by certain optimizations. Tail call optimization is a notable example of that: What is tail call optimization? There's nothing we can do about it.
C++23 <stacktrace>
This method was previously mentioned at: https://stackoverflow.com/a/69384663/895245 please consider upvoting that answer.
This is the best solution... it's portable, fast, shows line numbers and demangles C++ symbols. This option will displace every other alternative as soon as it becomes more widely available, with the exception perhaps only of GDB for one-offs without the need or recompilation.
cpp20_stacktrace/mystacktrace.h
#include <iostream>
#include <stacktrace>
void print_stacktrace() {
std::cout << std::stacktrace::current();
}
GCC 12.1.0 from Ubuntu 22.04 does not have support compiled in, so for now I built it from source as per: How to edit and re-build the GCC libstdc++ C++ standard library source? and set --enable-libstdcxx-backtrace=yes, and it worked!
Compile with:
g++ -O0 -ggdb3 -Wall -Wextra -pedantic -std=c++23 -o cpp20_stacktrace.out main.cpp -lstdc++_libbacktrace
Sample output:
0# print_stacktrace() at cpp20_stacktrace/mystacktrace.h:5
1# my_func_2() at /home/ciro/main.cpp:6
2# my_func_1(int) at /home/ciro/main.cpp:16
3# at /home/ciro/main.cpp:27
4# at :0
5# at :0
6# at :0
7#
If we try to use GCC 12.1.0 from Ubuntu 22.04:
sudo apt install g++-12
g++-12 -ggdb3 -O2 -std=c++23 -Wall -Wextra -pedantic -o stacktrace.out stacktrace.cpp -lstdc++_libbacktrace
It fails with:
stacktrace.cpp: In function ‘void my_func_2()’:
stacktrace.cpp:6:23: error: ‘std::stacktrace’ has not been declared
6 | std::cout << std::stacktrace::current();
| ^~~~~~~~~~
Checking build options with:
g++-12 -v
does not show:
--enable-libstdcxx-backtrace=yes
so it wasn't compiled in. Bibliography:
How to use <stacktrace> in GCC trunk?
How can I generate a C++23 stacktrace with GCC 12.1?
It does not fail on the include because the header file:
/usr/include/c++/12
has a feature check:
#if __cplusplus > 202002L && _GLIBCXX_HAVE_STACKTRACE
Boost stacktrace
The library has changed quite a lot around Ubuntu 22.04, so make sure your version matches: Boost stack-trace not showing function names and line numbers
The library is pretty much superseded by the more portable C++23 implementation, but remains a very good option for those that are not at that standard version yet, but already have a "Boost clearance".
Documented at: https://www.boost.org/doc/libs/1_66_0/doc/html/stacktrace/getting_started.html#stacktrace.getting_started.how_to_print_current_call_stack
Tested on Ubuntu 22.04, boost 1.74.0, you should do:
boost_stacktrace/mystacktrace.h
#include <iostream>
#define BOOST_STACKTRACE_LINK
#include <boost/stacktrace.hpp>
void print_stacktrace(void) {
std::cout << boost::stacktrace::stacktrace();
}
On Ubuntu 19.10 boost 1.67.0 to get the line numbers we had to instead:
#include <iostream>
#define BOOST_STACKTRACE_USE_ADDR2LINE
#include <boost/stacktrace.hpp>
void print_stacktrace(void) {
std::cout << boost::stacktrace::stacktrace();
}
which would call out to the addr2line executable and be 1000x slower than the newer Boost version.
The package libboost-stacktrace-dev did not exist at all on Ubuntu 16.04.
The rest of this section considers only the Ubuntu 22.04, boost 1.74 behaviour.
Compile:
sudo apt-get install libboost-stacktrace-dev
g++ -O0 -ggdb3 -Wall -Wextra -pedantic -std=c++11 -o boost_stacktrace.out main.cpp -lboost_stacktrace_backtrace
Sample output:
0# print_stacktrace() at boost_stacktrace/mystacktrace.h:7
1# my_func_2() at /home/ciro/main.cpp:7
2# my_func_1(int) at /home/ciro/main.cpp:17
3# main at /home/ciro/main.cpp:26
4# __libc_start_call_main at ../sysdeps/nptl/libc_start_call_main.h:58
5# __libc_start_main at ../csu/libc-start.c:379
6# _start in ./boost_stacktrace.out
Note that the lines are off by one line. It was suggested in the comments that this is because the following instruction address is being considered.
Boost stacktrace header only
What the BOOST_STACKTRACE_LINK does is to require -lboost_stacktrace_backtrace at link time, so we imagine without that it will just work. This would be a good option for devs who don't have the "Boost clearance" to just add as one offs to debug.
TODO unfortunately it didn't so well for me:
#include <iostream>
#include <boost/stacktrace.hpp>
void print_stacktrace(void) {
std::cout << boost::stacktrace::stacktrace();
}
then:
g++ -O0 -ggdb3 -Wall -Wextra -pedantic -std=c++11 -o boost_stacktrace_header_only.out main.cpp
contains the overly short output:
0# 0x000055FF74AFB601 in ./boost_stacktrace_header_only.out
1# 0x000055FF74AFB66C in ./boost_stacktrace_header_only.out
2# 0x000055FF74AFB69C in ./boost_stacktrace_header_only.out
3# 0x000055FF74AFB6F7 in ./boost_stacktrace_header_only.out
4# 0x00007F0176E7BD90 in /lib/x86_64-linux-gnu/libc.so.6
5# __libc_start_main in /lib/x86_64-linux-gnu/libc.so.6
6# 0x000055FF74AFB4E5 in ./boost_stacktrace_header_only.out
which we can't even use with addr2line. Maybe we have to pass some other define from: https://www.boost.org/doc/libs/1_80_0/doc/html/stacktrace/configuration_and_build.html ?
Tested on Ubuntu 22.04. boost 1.74.
Boost boost::stacktrace::safe_dump_to
This is an interesting alternative to boost::stacktrace::stacktrace as it writes the stack trace in a async signal safe manner to a file, which makes it a good option for automatically dumping stack traces on segfaults which is a super common use case: How to automatically generate a stacktrace when my program crashes
Documented at: https://www.boost.org/doc/libs/1_70_0/doc/html/boost/stacktrace/safe_dump_1_3_38_7_6_2_1_6.html
TODO get it to work. All I see each time is a bunch of random bytes. My attempt:
boost_stacktrace_safe/mystacktrace.h
#include <unistd.h>
#define BOOST_STACKTRACE_LINK
#include <boost/stacktrace.hpp>
void print_stacktrace(void) {
boost::stacktrace::safe_dump_to(0, 1024, STDOUT_FILENO);
}
Sample output:
1[FU1[FU"2[FU}2[FUm1#n10[FU
Changes drastically each time, suggesting it is random memory addresses.
Tested on Ubuntu 22.04, boost 1.74.0.
glibc backtrace
This method is quite portable as it comes with glibc itself. Documented at: https://www.gnu.org/software/libc/manual/html_node/Backtraces.html
Tested on Ubuntu 22.04, glibc 2.35.
glibc_backtrace_symbols_fd/mystacktrace.h
#include <execinfo.h> /* backtrace, backtrace_symbols_fd */
#include <unistd.h> /* STDOUT_FILENO */
void print_stacktrace(void) {
size_t size;
enum Constexpr { MAX_SIZE = 1024 };
void *array[MAX_SIZE];
size = backtrace(array, MAX_SIZE);
backtrace_symbols_fd(array, size, STDOUT_FILENO);
}
Compile with:
g++ -O0 -ggdb3 -Wall -Wextra -pedantic -rdynamic -std=c++11 -o glibc_backtrace_symbols_fd.out main.cpp
Sample output with -rdynamic:
./glibc_backtrace_symbols.out(_Z16print_stacktracev+0x47) [0x556e6a131230]
./glibc_backtrace_symbols.out(_Z9my_func_2v+0xd) [0x556e6a1312d6]
./glibc_backtrace_symbols.out(_Z9my_func_1i+0x14) [0x556e6a131306]
./glibc_backtrace_symbols.out(main+0x58) [0x556e6a131361]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f175e7bdd90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7f175e7bde40]
./glibc_backtrace_symbols.out(_start+0x25) [0x556e6a131125]
Sample output without -rdynamic:
./glibc_backtrace_symbols_fd_no_rdynamic.out(+0x11f0)[0x556bd40461f0]
./glibc_backtrace_symbols_fd_no_rdynamic.out(+0x123c)[0x556bd404623c]
./glibc_backtrace_symbols_fd_no_rdynamic.out(+0x126c)[0x556bd404626c]
./glibc_backtrace_symbols_fd_no_rdynamic.out(+0x12c7)[0x556bd40462c7]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f0da2b70d90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f0da2b70e40]
./glibc_backtrace_symbols_fd_no_rdynamic.out(+0x10e5)[0x556bd40460e5]
To get the line numbers without -rdynamic we can use addr2line:
addr2line -C -e glibc_backtrace_symbols_fd_no_rdynamic.out 0x11f0 0x123c 0x126c 0x12c7
addr2line cannot unfortunately handle the function name + offset in function format of when we are not using -rdynamic, e.g. _Z9my_func_2v+0xd.
GDB can however:
gdb -nh -batch -ex 'info line *(_Z9my_func_2v+0xd)' -ex 'info line *(_Z9my_func_1i+0x14)' glibc_backtrace_symbols.out
Line 7 of "main.cpp" starts at address 0x12d6 <_Z9my_func_2v+13> and ends at 0x12d9 <_Z9my_func_1d>.
Line 17 of "main.cpp" starts at address 0x1306 <_Z9my_func_1i+20> and ends at 0x1309 <main(int, char**)>.
A helper to make it more bearable:
addr2lines() (
perl -ne '$m = s/(.*).*\(([^)]*)\).*/gdb -nh -q -batch -ex "info line *\2" \1/;print $_ if $m' | bash
)
Usage:
xsel -b | addr2lines
glibc backtrace_symbols
A version of backtrace_symbols_fd that returns a string rather than printing to a file handle.
glibc_backtrace_symbols/mystacktrace.h
#include <execinfo.h> /* backtrace, backtrace_symbols */
#include <stdio.h> /* printf */
void print_stacktrace(void) {
char **strings;
size_t i, size;
enum Constexpr { MAX_SIZE = 1024 };
void *array[MAX_SIZE];
size = backtrace(array, MAX_SIZE);
strings = backtrace_symbols(array, size);
for (i = 0; i < size; i++)
printf("%s\n", strings[i]);
free(strings);
}
glibc backtrace with C++ demangling hack 1: -export-dynamic + dladdr
I couldn't find a simple way to automatically demangle C++ symbols with glibc backtrace.
https://panthema.net/2008/0901-stacktrace-demangled/
https://gist.github.com/fmela/591333/c64f4eb86037bb237862a8283df70cdfc25f01d3
Adapted from: https://gist.github.com/fmela/591333/c64f4eb86037bb237862a8283df70cdfc25f01d3
This is a "hack" because it requires changing the ELF with -export-dynamic.
glibc_ldl.cpp
#include <dlfcn.h> // for dladdr
#include <cxxabi.h> // for __cxa_demangle
#include <cstdio>
#include <string>
#include <sstream>
#include <iostream>
// This function produces a stack backtrace with demangled function & method names.
std::string backtrace(int skip = 1)
{
void *callstack[128];
const int nMaxFrames = sizeof(callstack) / sizeof(callstack[0]);
char buf[1024];
int nFrames = backtrace(callstack, nMaxFrames);
char **symbols = backtrace_symbols(callstack, nFrames);
std::ostringstream trace_buf;
for (int i = skip; i < nFrames; i++) {
Dl_info info;
if (dladdr(callstack[i], &info)) {
char *demangled = NULL;
int status;
demangled = abi::__cxa_demangle(info.dli_sname, NULL, 0, &status);
std::snprintf(
buf,
sizeof(buf),
"%-3d %*p %s + %zd\n",
i,
(int)(2 + sizeof(void*) * 2),
callstack[i],
status == 0 ? demangled : info.dli_sname,
(char *)callstack[i] - (char *)info.dli_saddr
);
free(demangled);
} else {
std::snprintf(buf, sizeof(buf), "%-3d %*p\n",
i, (int)(2 + sizeof(void*) * 2), callstack[i]);
}
trace_buf << buf;
std::snprintf(buf, sizeof(buf), "%s\n", symbols[i]);
trace_buf << buf;
}
free(symbols);
if (nFrames == nMaxFrames)
trace_buf << "[truncated]\n";
return trace_buf.str();
}
void my_func_2(void) {
std::cout << backtrace() << std::endl;
}
void my_func_1(double f) {
(void)f;
my_func_2();
}
void my_func_1(int i) {
(void)i;
my_func_2();
}
int main() {
my_func_1(1);
my_func_1(2.0);
}
Compile and run:
g++ -fno-pie -ggdb3 -O0 -no-pie -o glibc_ldl.out -std=c++11 -Wall -Wextra \
-pedantic-errors -fpic glibc_ldl.cpp -export-dynamic -ldl
./glibc_ldl.out
output:
1 0x40130a my_func_2() + 41
./glibc_ldl.out(_Z9my_func_2v+0x29) [0x40130a]
2 0x40139e my_func_1(int) + 16
./glibc_ldl.out(_Z9my_func_1i+0x10) [0x40139e]
3 0x4013b3 main + 18
./glibc_ldl.out(main+0x12) [0x4013b3]
4 0x7f7594552b97 __libc_start_main + 231
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f7594552b97]
5 0x400f3a _start + 42
./glibc_ldl.out(_start+0x2a) [0x400f3a]
1 0x40130a my_func_2() + 41
./glibc_ldl.out(_Z9my_func_2v+0x29) [0x40130a]
2 0x40138b my_func_1(double) + 18
./glibc_ldl.out(_Z9my_func_1d+0x12) [0x40138b]
3 0x4013c8 main + 39
./glibc_ldl.out(main+0x27) [0x4013c8]
4 0x7f7594552b97 __libc_start_main + 231
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f7594552b97]
5 0x400f3a _start + 42
./glibc_ldl.out(_start+0x2a) [0x400f3a]
Tested on Ubuntu 18.04.
glibc backtrace with C++ demangling hack 2: parse backtrace output
Shown at: https://panthema.net/2008/0901-stacktrace-demangled/
This is a hack because it requires parsing.
TODO get it to compile and show it here.
GDB scripting
We can also do this with GDB without recompiling by using: How to do an specific action when a certain breakpoint is hit in GDB?
We setup an empty backtrace function for our testing:
gdb/mystacktrace.h
void print_stacktrace(void) {}
and then with:
main.gdb
start
break print_stacktrace
commands
silent
backtrace
printf "\n"
continue
end
continue
we can run:
gdb -nh -batch -x main.gdb --args gdb.out
Sample output:
Temporary breakpoint 1 at 0x11a7: file main.cpp, line 21.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Temporary breakpoint 1, main (argc=1, argv=0x7fffffffc3e8) at main.cpp:21
warning: Source file is more recent than executable.
21 if (argc > 1) {
Breakpoint 2 at 0x555555555151: file gdb/mystacktrace.h, line 1.
#0 print_stacktrace () at gdb/mystacktrace.h:1
#1 0x0000555555555161 in my_func_2 () at main.cpp:6
#2 0x0000555555555191 in my_func_1 (i=1) at main.cpp:16
#3 0x00005555555551ec in main (argc=1, argv=0x7fffffffc3e8) at main.cpp:27
[Inferior 1 (process 165453) exited normally]
The above can be made more usable with the following Bash function:
gdbbt() (
tmpfile=$(mktemp /tmp/gdbbt.XXXXXX)
fn="$1"
shift
printf '%s' "
start
break $fn
commands
silent
backtrace
printf \"\n\"
continue
end
continue
" > "$tmpfile"
gdb -nh -batch -x "$tmpfile" -args "$#"
rm -f "$tmpfile"
)
Usage:
gdbbt print_stacktrace gdb.out 2
I don't know how to make commands with -ex without the temporary file: Problems adding a breakpoint with commands from command line with ex command
Tested in Ubuntu 22.04, GDB 12.0.90.
GDB code injection
TODO this is the dream! It might allow for both compiled-liked speeds, but without the need to recompile! Either:
with compile code + one of the other options, ideally C++23 <stacktrace>: How to call assembly in gdb? Might already be possible. But compile code is mega-quirky so I'm lazy to even try
a built-in dbt command analogous to dprintf dynamic printf: How to do an specific action when a certain breakpoint is hit in GDB?
libunwind
TODO does this have any advantage over glibc backtrace? Very similar output, also requires modifying the build command, but not part of glibc so requires an extra package installation.
Code adapted from: https://eli.thegreenplace.net/2015/programmatic-access-to-the-call-stack-in-c/
main.c
/* This must be on top. */
#define _XOPEN_SOURCE 700
#include <stdio.h>
#include <stdlib.h>
/* Paste this on the file you want to debug. */
#define UNW_LOCAL_ONLY
#include <libunwind.h>
#include <stdio.h>
void print_trace() {
char sym[256];
unw_context_t context;
unw_cursor_t cursor;
unw_getcontext(&context);
unw_init_local(&cursor, &context);
while (unw_step(&cursor) > 0) {
unw_word_t offset, pc;
unw_get_reg(&cursor, UNW_REG_IP, &pc);
if (pc == 0) {
break;
}
printf("0x%lx:", pc);
if (unw_get_proc_name(&cursor, sym, sizeof(sym), &offset) == 0) {
printf(" (%s+0x%lx)\n", sym, offset);
} else {
printf(" -- error: unable to obtain symbol name for this frame\n");
}
}
puts("");
}
void my_func_3(void) {
print_trace();
}
void my_func_2(void) {
my_func_3();
}
void my_func_1(void) {
my_func_3();
}
int main(void) {
my_func_1(); /* line 46 */
my_func_2(); /* line 47 */
return 0;
}
Compile and run:
sudo apt-get install libunwind-dev
gcc -fno-pie -ggdb3 -O3 -no-pie -o main.out -std=c99 \
-Wall -Wextra -pedantic-errors main.c -lunwind
Either #define _XOPEN_SOURCE 700 must be on top, or we must use -std=gnu99:
Is the type `stack_t` no longer defined on linux?
Glibc - error in ucontext.h, but only with -std=c11
Run:
./main.out
Output:
0x4007db: (main+0xb)
0x7f4ff50aa830: (__libc_start_main+0xf0)
0x400819: (_start+0x29)
0x4007e2: (main+0x12)
0x7f4ff50aa830: (__libc_start_main+0xf0)
0x400819: (_start+0x29)
and:
addr2line -e main.out 0x4007db 0x4007e2
gives:
/home/ciro/main.c:34
/home/ciro/main.c:49
With -O0:
0x4009cf: (my_func_3+0xe)
0x4009e7: (my_func_1+0x9)
0x4009f3: (main+0x9)
0x7f7b84ad7830: (__libc_start_main+0xf0)
0x4007d9: (_start+0x29)
0x4009cf: (my_func_3+0xe)
0x4009db: (my_func_2+0x9)
0x4009f8: (main+0xe)
0x7f7b84ad7830: (__libc_start_main+0xf0)
0x4007d9: (_start+0x29)
and:
addr2line -e main.out 0x4009f3 0x4009f8
gives:
/home/ciro/main.c:47
/home/ciro/main.c:48
Tested on Ubuntu 16.04, GCC 6.4.0, libunwind 1.1.
libunwind with C++ name demangling
Code adapted from: https://eli.thegreenplace.net/2015/programmatic-access-to-the-call-stack-in-c/
unwind.cpp
#define UNW_LOCAL_ONLY
#include <cxxabi.h>
#include <libunwind.h>
#include <cstdio>
#include <cstdlib>
#include <iostream>
void backtrace() {
unw_cursor_t cursor;
unw_context_t context;
// Initialize cursor to current frame for local unwinding.
unw_getcontext(&context);
unw_init_local(&cursor, &context);
// Unwind frames one by one, going up the frame stack.
while (unw_step(&cursor) > 0) {
unw_word_t offset, pc;
unw_get_reg(&cursor, UNW_REG_IP, &pc);
if (pc == 0) {
break;
}
std::printf("0x%lx:", pc);
char sym[256];
if (unw_get_proc_name(&cursor, sym, sizeof(sym), &offset) == 0) {
char* nameptr = sym;
int status;
char* demangled = abi::__cxa_demangle(sym, nullptr, nullptr, &status);
if (status == 0) {
nameptr = demangled;
}
std::printf(" (%s+0x%lx)\n", nameptr, offset);
std::free(demangled);
} else {
std::printf(" -- error: unable to obtain symbol name for this frame\n");
}
}
}
void my_func_2(void) {
backtrace();
std::cout << std::endl; // line 43
}
void my_func_1(double f) {
(void)f;
my_func_2();
}
void my_func_1(int i) {
(void)i;
my_func_2();
} // line 54
int main() {
my_func_1(1);
my_func_1(2.0);
}
Compile and run:
sudo apt-get install libunwind-dev
g++ -fno-pie -ggdb3 -O0 -no-pie -o unwind.out -std=c++11 \
-Wall -Wextra -pedantic-errors unwind.cpp -lunwind -pthread
./unwind.out
Output:
0x400c80: (my_func_2()+0x9)
0x400cb7: (my_func_1(int)+0x10)
0x400ccc: (main+0x12)
0x7f4c68926b97: (__libc_start_main+0xe7)
0x400a3a: (_start+0x2a)
0x400c80: (my_func_2()+0x9)
0x400ca4: (my_func_1(double)+0x12)
0x400ce1: (main+0x27)
0x7f4c68926b97: (__libc_start_main+0xe7)
0x400a3a: (_start+0x2a)
and then we can find the lines of my_func_2 and my_func_1(int) with:
addr2line -e unwind.out 0x400c80 0x400cb7
which gives:
/home/ciro/test/unwind.cpp:43
/home/ciro/test/unwind.cpp:54
TODO: why are the lines off by one?
Tested on Ubuntu 18.04, GCC 7.4.0, libunwind 1.2.1.
Linux kernel
How to print the current thread stack trace inside the Linux kernel?
libdwfl
This was originally mentioned at: https://stackoverflow.com/a/60713161/895245 and it might be the best method, but I have to benchmark a bit more, but please go upvote that answer.
TODO: I tried to minimize the code in that answer, which was working, to a single function, but it is segfaulting, let me know if anyone can find why.
dwfl.cpp: answer reached 30k chars and this was the easiest cut: https://gist.github.com/cirosantilli/f1dd3ee5d324b9d24e40f855723544ac
Compile and run:
sudo apt install libdw-dev libunwind-dev
g++ -fno-pie -ggdb3 -O0 -no-pie -o dwfl.out -std=c++11 -Wall -Wextra -pedantic-errors dwfl.cpp -ldw -lunwind
./dwfl.out
We also need libunwind as that makes results more correct. If you do without it, it runs, but you will see that some of the lines are a bit wrong.
Output:
0: 0x402b72 stacktrace[abi:cxx11]() at /home/ciro/test/dwfl.cpp:65
1: 0x402cda my_func_2() at /home/ciro/test/dwfl.cpp:100
2: 0x402d76 my_func_1(int) at /home/ciro/test/dwfl.cpp:111
3: 0x402dd1 main at /home/ciro/test/dwfl.cpp:122
4: 0x7ff227ea0d8f __libc_start_call_main at ../sysdeps/nptl/libc_start_call_main.h:58
5: 0x7ff227ea0e3f __libc_start_main##GLIBC_2.34 at ../csu/libc-start.c:392
6: 0x402534 _start at ../csu/libc-start.c:-1
0: 0x402b72 stacktrace[abi:cxx11]() at /home/ciro/test/dwfl.cpp:65
1: 0x402cda my_func_2() at /home/ciro/test/dwfl.cpp:100
2: 0x402d5f my_func_1(double) at /home/ciro/test/dwfl.cpp:106
3: 0x402de2 main at /home/ciro/test/dwfl.cpp:123
4: 0x7ff227ea0d8f __libc_start_call_main at ../sysdeps/nptl/libc_start_call_main.h:58
5: 0x7ff227ea0e3f __libc_start_main##GLIBC_2.34 at ../csu/libc-start.c:392
6: 0x402534 _start at ../csu/libc-start.c:-1
Benchmark run:
g++ -fno-pie -ggdb3 -O3 -no-pie -o dwfl.out -std=c++11 -Wall -Wextra -pedantic-errors dwfl.cpp -ldw
time ./dwfl.out 1000 >/dev/null
Output:
real 0m3.751s
user 0m2.822s
sys 0m0.928s
So we see that this method is 10x faster than Boost's stacktrace, and might therefore be applicable to more use cases.
Tested in Ubuntu 22.04 amd64, libdw-dev 0.186, libunwind 1.3.2.
libbacktrace
https://github.com/ianlancetaylor/libbacktrace
Considering the harcore library author, it is worth trying this out, maybe it is The One. TODO check it out.
A C library that may be linked into a C/C++ program to produce symbolic backtraces
As of October 2020, libbacktrace supports ELF, PE/COFF, Mach-O, and XCOFF executables with DWARF debugging information. In other words, it supports GNU/Linux, *BSD, macOS, Windows, and AIX. The library is written to make it straightforward to add support for other object file and debugging formats.
The library relies on the C++ unwind API defined at https://itanium-cxx-abi.github.io/cxx-abi/abi-eh.html This API is provided by GCC and clang.
See also
How can one grab a stack trace in C?
How to make backtrace()/backtrace_symbols() print the function names?
Is there a portable/standard-compliant way to get filenames and linenumbers in a stack trace?
Best way to invoke gdb from inside program to print its stacktrace?
automatic stack trace on failure:
on C++ exception: C++ display stack trace on exception
generic: How to automatically generate a stacktrace when my program crashes
For a linux-only solution you can use backtrace(3) that simply returns an array of void * (in fact each of these point to the return address from the corresponding stack frame). To translate these to something of use, there's backtrace_symbols(3).
Pay attention to the notes section in backtrace(3):
The symbol names may be unavailable
without the use of special linker
options.
For systems using the GNU linker, it is necessary to use the
-rdynamic linker
option. Note that names of "static" functions are not exposed,
and won't be
available in the backtrace.
In C++23, there will be <stacktrace>, and then you can do:
#include <stacktrace>
/* ... */
std::cout << std::stacktrace::current();
Further details:
  • https://en.cppreference.com/w/cpp/header/stacktrace
  • https://en.cppreference.com/w/cpp/utility/basic_stacktrace/operator_ltlt
Is there any way to dump the call stack in a running process in C or C++ every time a certain function is called?
You can use a macro function instead of return statement in the specific function.
For example, instead of using return,
int foo(...)
{
if (error happened)
return -1;
... do something ...
return 0
}
You can use a macro function.
#include "c-callstack.h"
int foo(...)
{
if (error happened)
NL_RETURN(-1);
... do something ...
NL_RETURN(0);
}
Whenever an error happens in a function, you will see Java-style call stack as shown below.
Error(code:-1) at : so_topless_ranking_server (sample.c:23)
Error(code:-1) at : nanolat_database (sample.c:31)
Error(code:-1) at : nanolat_message_queue (sample.c:39)
Error(code:-1) at : main (sample.c:47)
Full source code is available here.
c-callstack at https://github.com/Nanolat
Linux specific, TLDR:
backtrace in glibc produces accurate stacktraces only when -lunwind is linked (undocumented platform-specific feature).
To output function name, source file and line number use #include <elfutils/libdwfl.h> (this library is documented only in its header file). backtrace_symbols and backtrace_symbolsd_fd are least informative.
On modern Linux your can get the stacktrace addresses using function backtrace. The undocumented way to make backtrace produce more accurate addresses on popular platforms is to link with -lunwind (libunwind-dev on Ubuntu 18.04) (see the example output below). backtrace uses function _Unwind_Backtrace and by default the latter comes from libgcc_s.so.1 and that implementation is most portable. When -lunwind is linked it provides a more accurate version of _Unwind_Backtrace but this library is less portable (see supported architectures in libunwind/src).
Unfortunately, the companion backtrace_symbolsd and backtrace_symbols_fd functions have not been able to resolve the stacktrace addresses to function names with source file name and line number for probably a decade now (see the example output below).
However, there is another method to resolve addresses to symbols and it produces the most useful traces with function name, source file and line number. The method is to #include <elfutils/libdwfl.h>and link with -ldw (libdw-dev on Ubuntu 18.04).
Working C++ example (test.cc):
#include <stdexcept>
#include <iostream>
#include <cassert>
#include <cstdlib>
#include <string>
#include <boost/core/demangle.hpp>
#include <execinfo.h>
#include <elfutils/libdwfl.h>
struct DebugInfoSession {
Dwfl_Callbacks callbacks = {};
char* debuginfo_path = nullptr;
Dwfl* dwfl = nullptr;
DebugInfoSession() {
callbacks.find_elf = dwfl_linux_proc_find_elf;
callbacks.find_debuginfo = dwfl_standard_find_debuginfo;
callbacks.debuginfo_path = &debuginfo_path;
dwfl = dwfl_begin(&callbacks);
assert(dwfl);
int r;
r = dwfl_linux_proc_report(dwfl, getpid());
assert(!r);
r = dwfl_report_end(dwfl, nullptr, nullptr);
assert(!r);
static_cast<void>(r);
}
~DebugInfoSession() {
dwfl_end(dwfl);
}
DebugInfoSession(DebugInfoSession const&) = delete;
DebugInfoSession& operator=(DebugInfoSession const&) = delete;
};
struct DebugInfo {
void* ip;
std::string function;
char const* file;
int line;
DebugInfo(DebugInfoSession const& dis, void* ip)
: ip(ip)
, file()
, line(-1)
{
// Get function name.
uintptr_t ip2 = reinterpret_cast<uintptr_t>(ip);
Dwfl_Module* module = dwfl_addrmodule(dis.dwfl, ip2);
char const* name = dwfl_module_addrname(module, ip2);
function = name ? boost::core::demangle(name) : "<unknown>";
// Get source filename and line number.
if(Dwfl_Line* dwfl_line = dwfl_module_getsrc(module, ip2)) {
Dwarf_Addr addr;
file = dwfl_lineinfo(dwfl_line, &addr, &line, nullptr, nullptr, nullptr);
}
}
};
std::ostream& operator<<(std::ostream& s, DebugInfo const& di) {
s << di.ip << ' ' << di.function;
if(di.file)
s << " at " << di.file << ':' << di.line;
return s;
}
void terminate_with_stacktrace() {
void* stack[512];
int stack_size = ::backtrace(stack, sizeof stack / sizeof *stack);
// Print the exception info, if any.
if(auto ex = std::current_exception()) {
try {
std::rethrow_exception(ex);
}
catch(std::exception& e) {
std::cerr << "Fatal exception " << boost::core::demangle(typeid(e).name()) << ": " << e.what() << ".\n";
}
catch(...) {
std::cerr << "Fatal unknown exception.\n";
}
}
DebugInfoSession dis;
std::cerr << "Stacktrace of " << stack_size << " frames:\n";
for(int i = 0; i < stack_size; ++i) {
std::cerr << i << ": " << DebugInfo(dis, stack[i]) << '\n';
}
std::cerr.flush();
std::_Exit(EXIT_FAILURE);
}
int main() {
std::set_terminate(terminate_with_stacktrace);
throw std::runtime_error("test exception");
}
Compiled on Ubuntu 18.04.4 LTS with gcc-8.3:
g++ -o test.o -c -m{arch,tune}=native -std=gnu++17 -W{all,extra,error} -g -Og -fstack-protector-all test.cc
g++ -o test -g test.o -ldw -lunwind
Outputs:
Fatal exception std::runtime_error: test exception.
Stacktrace of 7 frames:
0: 0x55f3837c1a8c terminate_with_stacktrace() at /home/max/src/test/test.cc:76
1: 0x7fbc1c845ae5 <unknown>
2: 0x7fbc1c845b20 std::terminate()
3: 0x7fbc1c845d53 __cxa_throw
4: 0x55f3837c1a43 main at /home/max/src/test/test.cc:103
5: 0x7fbc1c3e3b96 __libc_start_main at ../csu/libc-start.c:310
6: 0x55f3837c17e9 _start
When no -lunwind is linked, it produces a less accurate stacktrace:
0: 0x5591dd9d1a4d terminate_with_stacktrace() at /home/max/src/test/test.cc:76
1: 0x7f3c18ad6ae6 <unknown>
2: 0x7f3c18ad6b21 <unknown>
3: 0x7f3c18ad6d54 <unknown>
4: 0x5591dd9d1a04 main at /home/max/src/test/test.cc:103
5: 0x7f3c1845cb97 __libc_start_main at ../csu/libc-start.c:344
6: 0x5591dd9d17aa _start
For comparison, backtrace_symbols_fd output for the same stacktrace is least informative:
/home/max/src/test/debug/gcc/test(+0x192f)[0x5601c5a2092f]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x92ae5)[0x7f95184f5ae5]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZSt9terminatev+0x10)[0x7f95184f5b20]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(__cxa_throw+0x43)[0x7f95184f5d53]
/home/max/src/test/debug/gcc/test(+0x1ae7)[0x5601c5a20ae7]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe6)[0x7f9518093b96]
/home/max/src/test/debug/gcc/test(+0x1849)[0x5601c5a20849]
In a production version (as well as C language version) you may like to make this code extra robust by replacing boost::core::demangle, std::string and std::cout with their underlying calls.
You can also override __cxa_throw to capture the stacktrace when an exception is thrown and print it when the exception is caught. By the time it enters catch block the stack has been unwound, so it is too late to call backtrace, and this is why the stack must be captured on throw which is implemented by function __cxa_throw. Note that in a multi-threaded program __cxa_throw can be called simultaneously by multiple threads, so that if it captures the stacktrace into a global array that must be thread_local.
You can also make the stack trace printing function async-signal safe, so that you can invoke it directly from your SIGSEGV, SIGBUS signal handlers (which should use their own stacks for robustness). Obtaining function name, source file and line number using libdwfl from a signal handler may fail because it is not async-signal safe or if the address space of the process has been substantially corrupted, but in practice it succeeds 99% of the time (I haven't seen it fail).
To summarize, a complete production-ready library for automatic stacktrace output should:
Capture the stacktrace on throw into thread-specific storage.
Automatically print the stacktrace on unhandled exceptions.
Print the stacktrace in async-signal-safe manner.
Provide a robust signal handler function which uses its own stack that prints the stacktrace in a async-signal-safe manner. The user can install this function as a signal handler for SIGSEGV, SIGBUS, SIGFPE, etc..
The signal handler may as well print the values of all CPU registers at the point of the fault from ucontext_t signal function argument (may be excluding vector registers), a-la Linux kernel oops log messages.
Another answer to an old thread.
When I need to do this, I usually just use system() and pstack
So something like this:
#include <sys/types.h>
#include <unistd.h>
#include <string>
#include <sstream>
#include <cstdlib>
void f()
{
pid_t myPid = getpid();
std::string pstackCommand = "pstack ";
std::stringstream ss;
ss << myPid;
pstackCommand += ss.str();
system(pstackCommand.c_str());
}
void g()
{
f();
}
void h()
{
g();
}
int main()
{
h();
}
This outputs
#0 0x00002aaaab62d61e in waitpid () from /lib64/libc.so.6
#1 0x00002aaaab5bf609 in do_system () from /lib64/libc.so.6
#2 0x0000000000400c3c in f() ()
#3 0x0000000000400cc5 in g() ()
#4 0x0000000000400cd1 in h() ()
#5 0x0000000000400cdd in main ()
This should work on Linux, FreeBSD and Solaris. I don't think that macOS has pstack or a simple equivalent, but this thread seems to have an alternative.
If you are using C, then you will need to use C string functions.
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
void f()
{
pid_t myPid = getpid();
/*
length of command 7 for 'pstack ', 7 for the PID, 1 for nul
*/
char pstackCommand[7+7+1];
sprintf(pstackCommand, "pstack %d", (int)myPid);
system(pstackCommand);
}
I've used 7 for the max number of digits in the PID, based on this post.
There is no standardized way to do that. For windows the functionality is provided in the DbgHelp library
You can use the Boost libraries to print the current callstack.
#include <boost/stacktrace.hpp>
// ... somewhere inside the `bar(int)` function that is called recursively:
std::cout << boost::stacktrace::stacktrace();
Man here: https://www.boost.org/doc/libs/1_65_1/doc/html/stacktrace.html
I know this thread is old, but I think it can be useful for other people. If you are using gcc, you can use its instrument features (-finstrument-functions option) to log any function call (entry and exit). Have a look at this for more information: http://hacktalks.blogspot.fr/2013/08/gcc-instrument-functions.html
You can thus for instance push and pop every calls into a stack, and when you want to print it, you just look at what you have in your stack.
I've tested it, it works perfectly and is very handy
UPDATE: you can also find information about the -finstrument-functions compile option in the GCC doc concerning the Instrumentation options: https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html
You can implement the functionality yourself:
Use a global (string)stack and at start of each function push the function name and such other values (eg parameters) onto this stack; at exit of function pop it again.
Write a function that will printout the stack content when it is called, and use this in the function where you want to see the callstack.
This may sound like a lot of work but is quite useful.
Of course the next question is: will this be enough ?
The main disadvantage of stack-traces is that why you have the precise function being called you do not have anything else, like the value of its arguments, which is very useful for debugging.
If you have access to gcc and gdb, I would suggest using assert to check for a specific condition, and produce a memory dump if it is not met. Of course this means the process will stop, but you'll have a full fledged report instead of a mere stack-trace.
If you wish for a less obtrusive way, you can always use logging. There are very efficient logging facilities out there, like Pantheios for example. Which once again could give you a much more accurate image of what is going on.
You can use Poppy for this. It is normally used to gather the stack trace during a crash but it can also output it for a running program as well.
Now here's the good part: it can output the actual parameter values for each function on the stack, and even local variables, loop counters, etc.
You can use the GNU profiler. It shows the call-graph as well! the command is gprof and you need to compile your code with some option.
Is there any way to dump the call stack in a running process in C or C++ every time a certain function is called?
No there is not, although platform-dependent solutions might exist.