When debuging a simple program in gdb, I want to continue the execution automatically after hitting breakpoints. As far as I know, there are two methods to accomplish it:
1) use hook-stop.
define hook-stop
continue
end
But it seems the hook-stop is trigged only once. When another breakpoint is hit next time, the execution still stops.
2) use gdb.events.stop.connect().
def handle_stop_event(event):
if isinstance(event, gdb.BreakpointEvent):
gdb.execute('continue')
gdb.events.stop.connect(handle_stop_event)
This method works well. But if there are too many breakpoints been hit, an error "Fatal Python error: Cannot recover from stack overflow." occurs.
It seems because of recursive call. I'm wondering why the gdb.execute('continue') would cause this issue.
I searched online and still didn't find a solution.
PS: gdb version 7.11.1 on Ubuntu 16.04
Any advice would be appreciate! Thanks in advance.
It seems, continue inside hook-stop doesn't work properly. Have you seen this question I posted yesterday?
I think, the best approach here is writing a convenience function in python and setting a conditional breakpoint. Or using commands — see the "Breakpoint Command Lists" section of the GDB user manual.
Here's how do to it (also described in the manual).
The python module:
import gdb
class should_skip_f(gdb.Function):
def __init__ (self):
super (should_skip_f, self).__init__("should_skip")
def invoke(self):
return True # Your condition here
should_skip_f()
(gdb) b <your target> if !$should_skip()
Or add the condition to existing breakpoints with
(gdb) condition <BNUM> !$should_skip()
The only downside is that you have to set the condition for each breakpoint individually, but that's scriptable. Also, I think, the commands syntax allows you to add commands to a list of breakpoints at once.
'commands [LIST...]'
'... COMMAND-LIST ...'
'end'
Specify a list of commands for the given breakpoints. The commands
themselves appear on the following lines. Type a line containing
just 'end' to terminate the commands.
As for the recursion — yeah, that's a bad "design" of a debugger script (if one should talk about design of one-off throwaway things). You can examine what happens there if you extend your python script like so
import inspect
...
def handle_stop_event(event):
...
print(len(inspect.stack())) # Or you can print the frames themselves...
The Python interpreter doesn't know that execution does not return from gdb.execute("continue"), so the Python stack frames for invocations of this function are never destroyed.
You can increase max stack size for the interpreter, but like I said, this script does not seem like the best solution to me.
Related
I am dealing with a large code base with tons of globals. Under some peculiar set of data it produces the wrong result. I wanted to automatically run few scenarios with gdb in automatic step-by-step execution and periodical dumping
of some values and recording the tracing in some file. Doing it manually will ruin my sight and my brain. I speculate that there is some globals mess-up. How to do this automatically? Use some scripting. All this is in RH linux.
Thanks in advance.
tried to do this manually using conditional breaks, but gave up after a while
I wanted to automatically run few scenarios with gdb in automatic step-by-step execution and periodical dumping of some values and recording the tracing in some file.
It may be significantly more effective to run the program under reverse debugger (such as rr), and trace the wrong result back to its source.
How to do this automatically?
You can't do automatically what you can't express as an algorithm, and you haven't described an algorithm you want to use. If it's something like "stop every 100 times foo is called and print the values of these 500 globals", than that's trivially automatable with GDB.
More complicated algorithms are possible with the use of embedded Python.
In the .gdbinit file in your home folder add
add-auto-load-safe-path /path_to_the_folder_containing_your_executable/
Now you can create another .gdbinit file in the same folder where your executable is that will be loaded when you start gdb from there (the .gdbinit file in your home that is also read - useful if you have nice stuff there such as loading pretty printers).
In this .gdbinit file, add the code below
file your_executable_name
start
# Optional
set args "<any command line parameters you program might need>"
# xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
# Add gdb commands below to set breakpoints, print variable values, etc
# xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
run
Gdb is very powerful and I'll list a few things that might help this automation.
You can set breakpoints with b filename.ext:line_number or b function_name
You can use commands breakpoint_number (see here) and then list commands that should be run after this breakpoint is hit. Use end to finish the commands
You know the breakpoint number, since they are created sequencially (note that the start will count as a breakpoint and thus the first one you add will be breakpoint 2)
You can use convenience variables to store useful things such as the address of an important object
The python api is very powerful
One idea is that you can save the address of the important global variables (if they are not always accessible) using convenience variables. For instance, add a breakpoint where this global variable is and then add to this breakpoint the command to save the address of this variable to a convenience variable followed by continue (with continue you will not see gdb stopping there).
set $var1 = &myglobal
continue
You might want to also delete this breakpoint with delete breakpoint_number before continue to avoid stopping at this breakpoint again.
Then as long is the object exists you can inspect it using p $var1 or p $var1->something when the program is stopped at a different breakpoint where myglobal might not be directly accessible.
In the commands that you add to be run when a breakpoint is hit you can do things such as echo message explaining where you are, p some_var to see the values of variables or even python some_complicated_python_code.
In the case you want to use python for more power it is worth reading the section about it in the manual. Let-me give you one example. Suppose one of your global variables was stored in a convenience variable called "$myvar". Then you can pass it to the python interpreter with
python myvar = gdb.parse_and_eval("$myvar")
You can also pass any variable in the current scope to parse_and_eval.
Now, suppose this global variable stores an object of a class with a "n_elem" attribute you want to check. You can print it with
python print(myvar["n_elem"])
You can also create a python file in the same folder and use
python from my_python_file import *
to import functions defined there.
With these gdb features you can pretty much automate whatever you might need.
Closed. This question needs to be more focused. It is not currently accepting answers.
Closed 8 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
What are your best tips for debugging Python?
Please don't just list a particular debugger without saying what it can actually do.
Related
What are good ways to make my Python code run first time? - This discusses minimizing errors
PDB
You can use the pdb module, insert pdb.set_trace() anywhere and it will function as a breakpoint.
>>> import pdb
>>> a="a string"
>>> pdb.set_trace()
--Return--
> <stdin>(1)<module>()->None
(Pdb) p a
'a string'
(Pdb)
To continue execution use c (or cont or continue).
It is possible to execute arbitrary Python expressions using pdb. For example, if you find a mistake, you can correct the code, then type a type expression to have the same effect in the running code
ipdb is a version of pdb for IPython. It allows the use of pdb with all the IPython features including tab completion.
It is also possible to set pdb to automatically run on an uncaught exception.
Pydb was written to be an enhanced version of Pdb. Benefits?
http://pypi.python.org/pypi/pudb, a full-screen, console-based Python debugger.
Its goal is to provide all the niceties of modern GUI-based debuggers in a more lightweight and keyboard-friendly package. PuDB allows you to debug code right where you write and test it – in a terminal. If you've worked with the excellent (but nowadays ancient) DOS-based Turbo Pascal or C tools, PuDB's UI might look familiar.
Nice for debugging standalone scripts, just run
python -m pudb.run my-script.py
If you are using pdb, you can define aliases for shortcuts. I use these:
# Ned's .pdbrc
# Print a dictionary, sorted. %1 is the dict, %2 is the prefix for the names.
alias p_ for k in sorted(%1.keys()): print "%s%-15s= %-80.80s" % ("%2",k,repr(%1[k]))
# Print the instance variables of a thing.
alias pi p_ %1.__dict__ %1.
# Print the instance variables of self.
alias ps pi self
# Print the locals.
alias pl p_ locals() local:
# Next and list, and step and list.
alias nl n;;l
alias sl s;;l
# Short cuts for walking up and down the stack
alias uu u;;u
alias uuu u;;u;;u
alias uuuu u;;u;;u;;u
alias uuuuu u;;u;;u;;u;;u
alias dd d;;d
alias ddd d;;d;;d
alias dddd d;;d;;d;;d
alias ddddd d;;d;;d;;d;;d
Logging
Python already has an excellent built-in logging module. You may want to use the logging template here.
The logging module lets you specify a level of importance; during debugging you can log everything, while during normal operation you might only log critical things. You can switch things off and on.
Most people just use basic print statements to debug, and then remove the print statements. It's better to leave them in, but disable them; then, when you have another bug, you can just re-enable everything and look your logs over.
This can be the best possible way to debug programs that need to do things quickly, such as networking programs that need to respond before the other end of the network connection times out and goes away. You might not have much time to single-step a debugger; but you can just let your code run, and log everything, then pore over the logs and figure out what's really happening.
EDIT: The original URL for the templates was: http://aymanh.com/python-debugging-techniques
This page is missing so I replaced it with a reference to the snapshot saved at archive.org: http://web.archive.org/web/20120819135307/http://aymanh.com/python-debugging-techniques
In case it disappears again, here are the templates I mentioned. This is code taken from the blog; I didn't write it.
import logging
import optparse
LOGGING_LEVELS = {'critical': logging.CRITICAL,
'error': logging.ERROR,
'warning': logging.WARNING,
'info': logging.INFO,
'debug': logging.DEBUG}
def main():
parser = optparse.OptionParser()
parser.add_option('-l', '--logging-level', help='Logging level')
parser.add_option('-f', '--logging-file', help='Logging file name')
(options, args) = parser.parse_args()
logging_level = LOGGING_LEVELS.get(options.logging_level, logging.NOTSET)
logging.basicConfig(level=logging_level, filename=options.logging_file,
format='%(asctime)s %(levelname)s: %(message)s',
datefmt='%Y-%m-%d %H:%M:%S')
# Your program goes here.
# You can access command-line arguments using the args variable.
if __name__ == '__main__':
main()
And here is his explanation of how to use the above. Again, I don't get the credit for this:
By default, the logging module prints critical, error and warning messages. To change this so that all levels are printed, use:
$ ./your-program.py --logging=debug
To send log messages to a file called debug.log, use:
$ ./your-program.py --logging-level=debug --logging-file=debug.log
It is possible to print what Python lines are executed (thanks Geo!). This has any number of applications, for example, you could modify it to check when particular functions are called or add something like ## make it only track particular lines.
code.interact takes you into a interactive console
import code; code.interact(local=locals())
If you want to be able to easily access your console history look at: "Can I have a history mechanism like in the shell?" (will have to look down for it).
Auto-complete can be enabled for the interpreter.
ipdb is like pdb, with the awesomeness of ipython.
print statements
Some people recommend a debug_print function instead of print for easy disabling
The pprint module is invaluable for complex structures
the obvious way to debug a script
python -m pdb script.py
useful when that script raises an exception
useful when using virtualenv and pdb command is not running with the venvs python version.
if you don't know exactly where that script is
python -m pdb ``which <python-script-name>``
PyDev
PyDev has a pretty good interactive debugger. It has watch expressions, hover-to-evaluate, thread and stack listings and (almost) all the usual amenities you expect from a modern visual debugger. You can even attach to a running process and do remote debugging.
Like other visual debuggers, though, I find it useful mostly for simple problems, or for very complicated problems after I've tried everything else. I still do most of the heavy lifting with logging.
If you are familiar with Visual Studio, Python Tools for Visual Studio is what you look for.
Winpdb is very nice, and contrary to its name it's completely cross-platform.
It's got a very nice prompt-based and GUI debugger, and supports remote debugging.
In Vim, I have these three bindings:
map <F9> Oimport rpdb2; rpdb2.start_embedded_debugger("asdf") #BREAK<esc>
map <F8> Ofrom nose.tools import set_trace; set_trace() #BREAK<esc>
map <F7> Oimport traceback, sys; traceback.print_exception(*sys.exc_info()) #TRACEBACK<esc>
rpdb2 is a Remote Python Debugger, which can be used with WinPDB, a solid graphical debugger. Because I know you'll ask, it can do everything I expect a graphical debugger to do :)
I use pdb from nose.tools so that I can debug unit tests as well as normal code.
Finally, the F7 mapping will print a traceback (similar to the kind you get when an exception bubbles to the top of the stack). I've found it really useful more than a few times.
Defining useful repr() methods for your classes (so you can see what an object is) and using repr() or "%r" % (...) or "...{0!r}..".format(...) in your debug messages/logs is IMHO a key to efficient debugging.
Also, the debuggers mentioned in other answers will make use of the repr() methods.
Getting a stack trace from a running Python application
There are several tricks here. These include
Breaking into an interpreter/printing a stack trace by sending a signal
Getting a stack trace out of an unprepared Python process
Running the interpreter with flags to make it useful for debugging
If you don't like spending time in debuggers (and don't appreciate poor usability of pdb command line interface), you can dump execution trace and analyze it later. For example:
python -m trace -t setup.py install > execution.log
This will dump all source line of setup.py install execution to execution.log.
To make it easier to customize trace output and write your own tracers, I put together some pieces of code into xtrace module (public domain).
When possible, I debug using M-x pdb in emacs for source level debugging.
There is a full online course called "Software Debugging" by Andreas Zeller on Udacity, packed with tips about debugging:
Course Summary
In this class you will learn how to debug programs systematically, how
to automate the debugging process and build several automated
debugging tools in Python.
Why Take This Course?
At the end of this course you will have a solid understanding about
systematic debugging, will know how to automate debugging and will
have built several functional debugging tools in Python.
Prerequisites and Requirements
Basic knowledge of programming and Python at the level of Udacity
CS101 or better is required. Basic understanding of Object-oriented
programming is helpful.
Highly recommended.
if you want a nice graphical way to print your call stack in a readable fashion, check out this utility: https://github.com/joerick/pyinstrument
Run from command line:
python -m pyinstrument myscript.py [args...]
Run as a module:
from pyinstrument import Profiler
profiler = Profiler()
profiler.start()
# code you want to profile
profiler.stop()
print(profiler.output_text(unicode=True, color=True))
Run with django:
Just add pyinstrument.middleware.ProfilerMiddleware to MIDDLEWARE_CLASSES, then add ?profile to the end of the request URL to activate the profiler.
I'm currently debugging syslinux (a boot loader) through the gdb stub of qemu.
Recently, I wrote some gdb commands that (un)load the debug symbols everytime a module is dynamically (un)loaded. In order not to disrupt the execution, I ended the commands with continue.
break com32/lib/sys/module/elf_module.c:282
commands
silent
python
name = gdb.parse_and_eval("module->name").string()
addr = int(str(gdb.parse_and_eval("module->base_addr")), 0)
gdb.execute("load-syslinux-module %s 0x%08x" % (name, addr))
end
continue
end
However, when stepping through the code line by line, if the next or step command makes the execution hit the breakpoint, the breakpoints takes precedence, the commands are executed, including the continue. And the execution continue irrespectively of the line-by-line debugging I was doign. This also happen if I try to step over the function that has this breakpoint.
How can I keep (un)loading the debug symbols on the fly while not interfering with the debugging?
Is there an alternative to the continue command? Maybe using breakpoints isn't the right way? I'd take any solution.
This can't be done from the gdb CLI. However, it is easy to do from Python.
In Python the simplest way is to define one's own gdb.Breakpoint subclass, and define the stop method on it. This method can do the work you like, then return False to tell gdb to continue.
The stop facility was designed to avoid the problems with cont in commands. See the documentation for more details.
I know the answer to this, I'm putting it up here for others to see it
If you use eclipse CDT, you probably understand that eclipse isn't a debugger, it's just an application front-end, specifically to GDB. So when debugging C++ programs, you're actually just using GDB in a more comfortable manner. If you ever have to debug a multithreaded program in eclipse CDT, you'll realize that things quickly get hectic because when you hit a breakpoint, all threads stop, and when one tries to execute a single line in a specific thread, it also runs the other threads. In order for it to work properly, the threads have to be able to be run arbitrarily and exlusively-so that when the programmer executes a single line, it only executes the specific thread.
So, by default, gdb's settings by default leave the "scheduler-locking" turned off. If you debug multithreaded applications you'll understand that this must be on in GDB in order for the desired behavior to be achieved. How does one run this command:
set scheduler-locking on
in GDB within eclipse CDT?
At least one way to do it that certainly solves the problem is knowing how to navigate the immense set of features that eclipse offers. Typically, when a program starts, eclipse CDT switches the console window (if you have it open, typically it's on the bottom) to show the input/output of the program.
But you can change this if you didn't know-see this image. That button on the second to last right-the blue one that looks like a monitor-you can select the GDB input console. It was discussed also in this thread.
From there merely type the command.
SOLVED, BUT NEED A BETTER SOLUTION
But now that this has been solved, to solve it in a better way as a matter of convience; having to type set scheduler-locking on every time a program starts is silly. But the problem with loading a gdbinit file is that the gdbinit file gets sourced before eclipse has set the program for gdb to solve. This is a problem, as it causes the debugger view to hang within eclipse, as gdb complains. To understand what is happening, try and fire up gdb, then give the command without loading a binary to execute. It fails-so how does one set this as an option that is sticky?
Maybe if you add the following gdb script which could set the variable when the program stops and turns it off if you continue:
define hook-step
set scheduler-locking on
end
define hookpost-step
set scheduler-locking off
end
define hook-run
set scheduler-locking off
end
define hook-continue
set scheduler-locking off
end
My answer is derived from the one by #user1448557 . Unfortunately, I don't currently have enough reputation to comment on it (or to upvote it by the way). The strategy seems great, but the answer might be a bit outdated because it doesn't involve "set scheduler-locking step". I have put the following in my gdb initialization file (within my Eclipse project) and it does what I want.
#inspired from [link to this thread][1]
define hookpost-run
set scheduler-locking step
end
With regards to the comment by #rbaleksandar, Eclipse CDT launch configurations allow one to specify a "GDB Command File" and the default is usually .gdbinit
I decided to find out how our C/C+ *nix practitioners use the gdb debugger.
Here is what I typically use:
b - break filename.c:line #, function, filename.cpp:function, className::Member
n, c, s -- next continue step
gdb program name => set breakpoints ==> run [parameter list] (I do this to set break points before the program starts)
l - to list the surrounding source code.
attach processID
6 break [location]
gdb programName corefile.core (to examine why app crashed)
I also sometimes set breakpoint at exit function (break exit) to examine program stacks
info b to examine all the breakpoints
clear [breakpoints list ]
How do you use it?
Besides things that have already been posted i also use:
a .gdbinit file for STL containers
signal SIGNAL noprint nostop for some custom signals that are of no real interest when debugging
C-Casts to dereference pointers
catchpoints (catch throw, catch catch)
condition for conditional break- and watchpoints
rarely gdbserver for remote debugging
gdb program coredump, for those embarassing segfaults ;)
PS: One reason i personally love gdb btw. is that it supports tab-completion for nearly everything (gdb commands, symbols in the symbol table, functions, memberfunctions etc.). This is a fairly good productivity boost in my opinion.
Scripting is a nice GDB feature.
First you set a breakpoint, like: b someFunction\n.
Then you run command: commands\n. GDB will ask for commands for that breakpoint.
Common scenario is to print some value and then continue, so you will enter: p someVar\n continue\n.
To end the script press: Ctrl-D
After running program you will see your script executed occasionally when the breakpoint occurs.
Most useful gdb commands in my opinion (aside from all already listed):
info threads - information about threads
thread N - switch to thread N
catch throw - break on any thrown exception. Useful when you caught the bug only after the stack unwound.
printf,print - examine any and all expressions, printf accepts C-style formatting specifiers
Finally, if debugging over a slow link, the text UI might be of use. To use it, start gdb with the --tui command-line switch.
gdb is not my speciality, but here is what i use:
bt list a stack
up, down moving in a stack
until continue until a line with greater number than current is reached -- for exiting loops
watch [expr] break the program when expr changes
... but mostly i use ddd as a frontend to gdb
Type Ctrl-X Ctrl-A to open a simple window with source preview.
Some time ago I found cgdb:
http://cgdb.sourceforge.net/
This is a curses (color console) based frontend for gdb that made my life a lot happier when I was restricted to debugging in a console window.
See the user guide at http://sources.redhat.com/gdb/current/onlinedocs/gdb_toc.html.
There are also a couple of uses that are not directly connected with debugging. For example it
can be used for C expression evaluation:
(gdb) printf "%lu\n", (unsigned long)(-3L)
4294967293
i use the gdb -tui switch for a great 'text user interface' (a kind of gui in text mode). It supports multiple windows and is generally much more friendly than using the 'list' command (since it shows the source in a sep window)
Beginners using gdb will feel it as tough. But there GUI based tool DDD(Data Display Debugger) which is same as gdb. u have a console in the bottom to run gdb commands and top 3/4th portion would be the code. U ll have the option to learn and understand the commands and the flow excatly