Here is a sample session directly from gdb console
Starting program:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Breakpoint 1, 0x00000000025654f0 in ~F()
(gdb) bt
#0 0x00000000025654f0 in ~F()
at hello.cpp:123
(gdb) c
Continuing.
foo.cpp:122:12: runtime error: member call on null pointer of type 'Object'
Here is my .gdbinit file
set pagination off
set language c++
set print pretty on
set logging file gdb.txt
set logging on
break ~F()
info breakpoints
r
bt
c
set logging off
quit
and the gdb.txt produced looks something like this:
Breakpoint 1 at 0x25654f0
Num Type Disp Enb Address What
1 breakpoint keep y 0x00000000025654f0 <~F()>
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Breakpoint 1, 0x00000000025654f0 in ~F()
#0 0x00000000025654f0 in ~F()
....
Breakpoint 1, 0x00000000025654f0 in ~F()
I don't see foo.cpp:122:12: runtime error: member call on null pointer of type 'Object' coming out in the log. How do I get that into my log?
Thanks
That message come from your program, not from gdb.
One way to make it work is to have your program and gdb write to the same log. The only trick here is to make sure they are both writing in "append" mode. There is a "set logging" subcommand for this, and for your program, you can run like:
(gdb) run >> log
Related
Environment
OS: CentOS 7.9_x64
Memory, CPU, current Disk Space:Memory 96G, Disk 1T
TDengine Version:TDengine-server-2.0.20.13-Linux-x64
TDengine taosd daemon coredump.
gdb output:
[New LWP 5461]
[New LWP 5499]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/usr/bin/taosd'.
Program terminated with signal 11, Segmentation fault.
#0 0x000056308db735cf in gcBuildQueryJson (pContext=0x7fdfdc0008c0, cmd=0x7fdfe00014a0, result=0x7fdfcc048ab0, numOfRows=682) at /home/ubuntu/workroom/jenkins/TDinternal/community/src/plugins/http/src/httpGcJson.c:154
154 /home/ubuntu/workroom/jenkins/TDinternal/community/src/plugins/http/src/httpGcJson.c: No such file or directory.
Missing separate debuginfos, use: debuginfo-install glibc-2.17-324.el7_9.x86_64
how to resolve it?
how to resolve it?
It's a bug in TDengine-server. You don't "resolve" bugs.
You can try to figure out what the bug is (via debugging), or you can try newer version of TDengine-server (current appears to be 2.2.0.2) and hope that the particular bug you've hit has been fixed.
I had attached gdb to a long running process(>25 hours). To manage the session, I used screen on my Ubuntu machine. I could get the session back. I got back the gdb console. But on continuing I saw my process throw SIGABRT and exit followed up by other process exit messages.
[New LWP 122]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007fe8ef29ea15 in futex_abstimed_wait_cancelable (private=0, abstime=0x7ffc8c628420, expected=0, futex_word=0x7fe8e6378640) at ../sysdeps/unix/sysv/
linux/futex-internal.h:205
205 ../sysdeps/unix/sysv/linux/futex-internal.h: No such file or directory.
(gdb) c
Continuing.
(gdb) [Thread 0x7be8d8bfd700 (LWP 48) exited]
Thread 32 "my-process" received signal SIGABRT, Aborted.
[Switching to Thread 0x7be8d2bbd700 (LWP 60)]
0x00007fe8eece4428 in __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
54 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) c
Continuing.
Couldn't get registers: No such process.
Couldn't get registers: No such process.
Couldn't get registers: No such process.
(gdb) [Thread 0x7be8b67ff700 (LWP 119) exited]
[Thread 0x7be8b49fe700 (LWP 122) exited]
...
I am not able to get the gdb console after that. Though I see a process running when I run ps -ef
root 133 0 1 Jan14 ? 00:26:09 gdb --pid=23
How do get back the console for this gdb process? I wanted to see the backtrace.
Or is there a better way to attach gdb to a long running process ?
We're running squid from with gdb - that way we can automatically generate backtraces for debugging.
backtrace=`mktemp`
gdb -q -x /etc/service/squid3/gdbcommands /usr/sbin/squid 2>&1 >$backtrace
/usr/bin/mail -s "`hostname`: Squid was restarted (backtrace)" someaddress#charite.de < backtracetrace
rm $backtrace
/etc/service/squid3/gdbcommands contains:
set args -NsYC
handle SIGPIPE pass nostop noprint
handle SIGTERM pass nostop noprint
handle SIGUSR1 pass nostop noprint
handle SIGHUP pass nostop noprint
handle SIGSEGV stop
handle SIGABRT stop
run
set print pretty
backtrace full
generate-core-file
quit
But, every now and then, squid is "just" being stopped & restarted, with no crash being involved at all. In that case I'm still getting an email containing:
Reading symbols from /usr/sbin/squid...done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[Inferior 1 (process 57867) exited normally]
/etc/service/squid3/gdbcommands:10: Error in sourced command file:
No stack.
(gdb) quit
And of course there's no stack, since the program exited ok.
How can I change my gdbcommands file to avoid this?
This can be done using either Python or the gdb CLI. Since the CLI is a bit simpler, when possible, I'll sketch that approach.
First, you might as well only create a core file on a bad exit. And, we'll use the gdb exit code later, so let's arrange for that to tell the calling script what happened.
Where your current script says:
backtrace full
generate-core-file
quit
... instead use:
if !$_isvoid($_exitsignal) || (!$_isvoid($_exitcode) && $_exitcode != 0))
backtrace full
generate-core-file
quit 0
end
quit 1
Then your calling script can check the exit code of gdb:
if gdb your args here; then
mail results
fi
I am compiling the unit tests (via GoogleTests) for my program and whenever I try to compile in DEBUG mode on Solaris 11.3 with Eigen 3.2.x, I'm getting this SIGSEGV error then core dump when running the program in gdb:
(gdb) r
...
[Thread debugging using libthread_db enabled] [New Thread 1 (LWP 1)]
Program received signal SIGSEGV, Segmentation fault. [Switching to
Thread 1 (LWP 1)] 0x0830fc30 in
Eigen::internal::ploadu (
from=0xfeffe5a0) at ./eigen/Eigen/src/Core/arch/SSE/Complex.h:307 307 {
EIGEN_DEBUG_UNALIGNED_LOAD return Packet1cd(ploadu((const
double*)from)); }
(gdb)
When print from in gdb this is what I'm getting:
gdb p from: (const std::complex< double > *) 0xfeffe5a0
This SIGSEGV only on Solaris, and only when compiling with -Og. I've compiled and tested it on multiple other OSes and there are no issues whatsoever. Is this a known issue? It looks it has to do with some SSE optimizations and alignments, however I cannot pinpoint what exactly is going on.
trace point can be traced, but trace actions does not work normally.
at the last of gdb side below shows trace point is traced.
but "collect $regs" does not work as expected.
my platform is RH6.4.
1. gdbserver side.
gdbserver :10000 ./a.out
Process ./a.out created; pid = 10466
Listening on port 10000
Remote debugging from host 127.0.0.1
2. gdb side.
gdb a.out
(gdb) target remote :10000
Remote debugging using :10000
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
0x00000033b7000b00 in _start () from /lib64/ld-linux-x86-64.so.2
Created trace state variable $trace_timestamp for target's variable 1.
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.107.el6.x86_64
(gdb) trace main
Tracepoint 1 at 0x400541: file a.c, line 12.
(gdb) actions 1
collect $regs
end
(gdb) tstart
(gdb) break 15
Breakpoint 2 at 0x40055f: file a.c, line 15.
Breakpoint 2, main (argc=1, argv=0x7fffca819f08) at a.c:18
18 sleep (1);
(gdb) cont
Continuing.
(gdb) tstop
(gdb) tfind
Found trace frame 0, tracepoint 1
12 c = 2;
I suppose you expected tracepoint to be in actual main declaration line in the source file, am I wrong?
The important part is that it is placed in function's entry point, this is, actually, first function's code line that, looking at the information you provided, it should be c = 2;
On the other hand, this is just a stupid detail, please note that you have no code at line 15 and breakpoint has been set at line 18.
Edit:
According to your comments, you expected tfind to dump all collected registers but you would need an extra step for this: by using tfind with no argument you selected next tracepoint (first one in this case) and, to dump this tracepoint's action collected info, you should call tdump