I'm trying to debug a server I wrote with gdb as it segfaults under very specific and rare conditions.
Is there any way I can make gdb run in the background (via quiet or batch mode?), follow children (as my server is a daemon and detaches from the main PID) and automatically dump the core and the backtrace (to a designated file) once the program crashes?
Assuming you have appropriate permissions, you can have gdb attach to any process. You can do it on the command line with:
gdb /path/to/binary _pid_
or from within gdb with the attach command:
attach _pid_
So, once your daemon has started, you can use either of these techniques to attach to the final PID your daemon is running as. Attaching gdb stops the process which you are tracing so you will need to issue a "continue" to restart it.
I don't know a direct way to get gdb to run arbitrary commands when the program crashes. Here is one workaround I can think of:
Create and register a signal handlers for SIGSEGV.
Tell gdb not to stop on that signal (handle SIGSEGV nostop)
Set a breakpoint at the first line of your signal handler.
Assign commands to the breakpoint from step 3
Why not just run the process interactively in a persistent screen session? Why must it be a daemon when debugging? Or just run gdb in the screen session and attach it to the running process (e.g. gdb /path/to/binary -p PID_of_binary) after it forks.
First, I'd setup your shell / environment to give you a core dump. In bash:
ulimit -c unlimited
Once you have the core dump, you can use gdb to examine the stack trace:
gdb /path/to/app /path/to/core/file
I'm not really a gdb expert but two things come to mind
Tracepoints which might give you the necessary information as your program runs or
Use gdb's remote debugging facility to debug your program while it's running as a daemon.
How to generate a stacktrace when my gcc C++ app crashes answer for this question should do what you want. (assuming you can make changes in your code)
You might want to take a look at how Samba facilitates debugging; it has a configurable "panic action" that can suspend the application, notify the developer, spawn gdb, etc., and is run as part of its signal handler. See lib/util/fault.c in the Samba source tree.
My practice: comment out daemon function call, rebuild binary, then use gdb to run.
Related
When running my program, I am getting Gtk-CRITICAL error on the Terminal, which I'd like to fix.
I found this link through Google, but turns out there is no such option in gdb:
igor#IgorReinCloud ~/dbhandler/Debug/dbhandler $ gdb --g-fatal-warnings dbhandler
gdb: unrecognized option '--g-fatal-warnings'
Use `gdb --help' for a complete list of options.
I also tried to set a breakpoint on the g_log() function, but the execution didn't stop there.
What am I missing?
Thank you.
you can use G_DEBUG=fatal-criticals, so that the application execution breaks at the first location where a critical is emitted.
To run inside gdb, run G_DEBUG=fatal-criticals gdb my-app, and as usual, do run inside gdb to run your application.
You may also set the G_DEBUG environment variable with export (if in bash). Thus you could do export G_DEBUG=fatal-criticals, and then run your app as gdb my-app, which will have the same effect.
See https://docs.gtk.org/glib/running.html for more details
What am I missing?
Looks like that after reading the link, you were confused that gdb should have --g-fatal-warnings option for debugging Gtk applications. This is not the case because gdb is not Gtk application, but your program is. So you should run your program with --g-fatal-warnings option inside gdb like this:
gdb --args dbhandler --g-fatal-warnings
See also related question How do I run a program with commandline args using gdb within a bash script?.
Why is gdb showing that the program exited during its startup, so before to stop at the first breakpoint in the main function ?
Some steps:
$ gdb --cd $programhome -tui -tty $reservedtty --args myprogram
b main
r
gdb shows:
Starting program: myprogram
During startup program exited with code 1.
I already tried to break at exit() function, without success.
Why is gdb exiting before to stop at the first breakpoint in the main function
GDB is not exiting. Your program does.
It does exit before reaching main.
This can happen for a few reasons, such as:
Corrupt binary -- the kernel rejects it in execve system call for some reason and not a single instruction of the program actually runs.
The dynamic linker rejects it (e.g. because some required library or symbol is missing)
Your shell refuses to execute the program (bad ~/.bashrc, bad $PATH, etc).
You can narrow down the actual cause by running the program outside GDB (does it run?), running without ~/.bashrc, using (gdb) catch syscall exit_group (on Linux), etc.
There was a permission issue accessing the secondary terminal port.
The gdb is being started with the parameter -tty which switches the input/output to another tty port (in that case pseudo: pts).
When the two terminals are opened by different users, that problem occurs, even if after the first logon you change the user with su command, the first user logged needed to be the same among the two ttys.
I want to debug a c program with gdb which is invoked by shell script. In this shell script , there are lot of things done and many environment variables are set.
This shell script is invoked by boost::process::launch from a c++ program.
I can change c++ program, shell script, and the c program itself, but can't change the architecture of this flow.
Is there any way , so that i can use gdb to debug the program.
If there is no solution, is there a way to dump all environment settings before launching shell script, so that i can launch same script with these settings to debug it later. I will prefer a portable and long term solution.
Two easy options:
Attach gdb after the program has started with gdb -p <pid of process> if it doesn't matter to stop it at a specific point.
Insert a raise(SIGSTOP); in the C program where you want it to stop. Once the process is stopped, attach gdb as in 1, set any breakpoints you need and then send the process a SIGCONT signal (kill -CONT <pid of process>) to cause it to continue.
I'm doing an OS class that's based on xv6 and I wrote a program that needs to run on it.
I know that I can debug kernel code with make qemu-gdb but I'm not sure how to debug my own user program.
Lets say I want to debug cat, how would I go about doing that?
Thanks
P.S. isn't there an xv6 tag? should this question even go here?
From the xv6 top-level dir:
Run the emulator in debug mode (assuming no X11): make qemu-nox-gdb
In other terminal just run the debugger loading the kernel symbols with:
gdb kernel This is important, otherwise the debugger will be confused between kernel and and user program symbols, for example main()
From the gdb interface run: (gdb) target remote localhost:26000
where 26000 is the TCP port that the step #1 report at the end (this might change).
Load the user exec with (gdb)file user_program
Place a breakpoint (gdb) break main and continue with (gdb) continue
etc...
file cat, break main, continue
semi reference running and debugging xv6
169 sigsuspend(&set);
(gdb)
As you see ,after executing syssuspend(&set); gdb will hang ,how can I have a chance to type any commands?
If you want to debug several processes running the same executable (a.out), you want the (relatively) new GDB multi-inferior support.