I am trying to debug a very sporadic issue in my application. If ran ~1000 times my application surely hits a certain line it shouldn't and I would like to view the stack.
I tried using a gdb script cmd.gdb for this:
set logging overwrite on
set pagination off
set $n = 1000
break file.c:496
while $n-- > 0
ignore 1 9
condition 1 global_var == 10
run
end
How should I modify this script in order to print the stack when the breakpoint is reached?
I tried adding this after "run":
if $_siginfo
bt
loop_break
end
but it doesn't seem to work.
Actually, I have a Github repo with a Python-GDB extension, which does exactly the same thing as You have described, but with some more functionality.
You can just clone the repo:
git clone https://github.com/Viaceslavus/gdb-debug-until.git
and feed the python script to GDB with the following command inside GDB:
source <python script path>
Then, according to your example, you should run the next command:
debug-until file.c:496 --args="" --var-eq="global_var:10" -r=1000
*some remarks:
file.c:496 here is a starting breakpoint
"--args" parameter contains the arguments for your program
"--var-eq" is a debugging event, where 'global_var' is a variable name and '10' is a value
and finally the "-r" option specifies the number of times the program will be ran.
So all together this command will run your program 1000 times and will immediately notify You when the 'global_var' will be equal to 10.
Any additional information about the project could be found here:
https://github.com/Viaceslavus/gdb-debug-until.git in the README file.
Related
GDB allows registering a set of commands to a specific breakpoint via commands NUM syntax. I need to register the set of commands for a specific breakpoint via an external file, by using a syntax something like the following:
commands ./main.c:18
silent
print buffer[0]
cont
end
commands ./io.c:29
silent
printf "Hello world %i\n", myvar1
cont
end
The commands path/to/file:XX syntax is made up by me. Because the NUM in commands NUM syntax requires exactly the breakpoint's runtime ID number (assigned by GDB), I can not use a deterministic syntax for that purpose.
I'm currently registering breakpoints via a text file with such a content:
break ./main.c:18
break ./io.c:29
and then issuing source breakpoints.txt command inside GDB. It seems that there is no way to register commands at the same time while registering a breakpoint:
(gdb) help break
Set breakpoint at specified line or function.
break [PROBE_MODIFIER] [LOCATION] [thread THREADNUM] [if CONDITION]
PROBE_MODIFIER shall be present if the command is to be placed in a
probe point. Accepted values are -probe' (for a generic, automatically guessed probe type), -probe-stap' (for a SystemTap probe) or
`-probe-dtrace' (for a DTrace probe).
LOCATION may be a line number, function name, or "*" and an address.
If a line number is specified, break at start of code for that line.
If a function is specified, break at start of code for that function.
If an address is specified, break at that exact address.
With no LOCATION, uses current execution address of the selected
stack frame. This is useful for breaking on return to a stack frame.
THREADNUM is the number from "info threads".
CONDITION is a boolean expression.
Multiple breakpoints at one place are permitted, and useful if their
conditions are different.
Question
Is there any easy way to set some predetermined commands for a predetermined breakpoint from within a file?
If not, is there any equivalent way to pass the (gdb) info breakpoints output to a file or a program while pipe is not available in GDB (version 5.3)? Currently I'm trying a workaround by using logging feature for that purpose:
set logging file /tmp/breakpoints
set logging on
info breakpoints
set logging off
Is there any easy way to set some predetermined commands for a predetermined breakpoint from within a file?
Yes: if you use commands without NUM, the commands will apply to the last breakpoint set. So you want something like:
break main.c:18
commands
silent
print buffer[0]
cont
end
I need to debug a flow using gdb - I do not know the call stack hence cannot set break points and going thru first entry points will be really very tedious in project code that runs thru thousands of line.
In same regards is there a way that when I start the program execution via gdb we enable some commands (after some initial breakpoint) - hence when the program starts processing further on it print file name line number without user interaction - something like code flow itself?
Well I want to list lines of code when executing via GDB - like we do
when breakpoint is set and we run 'step'.
You can run step in infinite loop like this:
(gdb) start
Temporary breakpoint 2, main () at ttt123.cpp:23
23 vector<A> v1;
(gdb) while 1
>step
>end
I have a program that needs to run a program we'll call externalProg in parallel on our linux (CentOS) cluster - or rather, it needs to run many instances of externalProg, each on different cores. Each "thread" creates 3 files based on a few parameters - the inputs to externalProg, a command file to tell externalProg how to execute my file, and a bash script to set up the environment (calls a setup script provided by the manufacturer) and actually call externalProg with my inputs.
Since this needs to be parallel with an unknown number of concurrent threads and I don't want to risk overwriting another thread's files, I am creating temp files using
mkstemp("PREFIX_XXXXXX")
for these input files. After the external program runs, I extract the relevant data and store it, and close the temp files (therefore deleting them).
We'll call the files created (Which actually have a name based on the template above)
tmpInputs - Inputs to externalProg
tmpCommand - Input that tells externalProg how to execute tmpInputs
tmpBash - bash script to set up and call externalProg with my inputs
The file tmpBash looks something like
source /path/to/setup/script # Sets up environment variables
externalProg < /path/to/tmpCommand
where tmpCommand is just a simple text file.
The problem I'm having is actually executing the bash script. Within my program, I call
ostringstream launchcmd;
launchcmd << "bash " << path_to_tmpBash
system(launchcmd.str().c_str());
But nothing happens. No error, no warning, no 'file not found' or permission denied or anything. I have verified that the files are being created and have the correct content. The rest of the code after system() is executed successfully (Though it fails since externalProg wasn't run).
Strangely, if I go back to the terminal and type
bash /path/to/tmpBash
then externalProg is executed successfully. I have also cout'd the launchcmd string, copy and pasted that in to the terminal, which also works successfully. For some reason, this only fails when called within my program.
After a bit of experimentation, I've determined that system() calls /bin/sh on our cluster. If I change launchcmd to look like
/path/to/tmpBash
(So that the full command should look like /bin/sh /path/to/tmpBash), I get a permission denied error, which is no surprise. The problem is that I can't chmod +x the tmpBash file while it's still open, and if I close the file, it gets deleted - so I'm not sure how to address that.
Is there something obviously wrong I'm doing, or does system() have some nuance that I'm missing?
edit: I wanted to add that I can successfully call things like
system("echo $PATH")
and get the expected results (in this case, my default $PATH).
Two separate ideas:
Change your SHELL environment variable to be /bin/bash, then call system(),
or:
Use execve directly `execve('/bin/bash', ['/path/to/tmpBash'], environ)
The go test command has support for the -c flag, described as follows:
-c Compile the test binary to pkg.test but do not run it.
(Where pkg is the last element of the package's import path.)
As far as I understand, generating a binary like this is the way to run it interactively using GDB. However, since the test binary is created by combining the source and test files temporarily in some /tmp/ directory, this is what happens when I run list in gdb:
Loading Go Runtime support.
(gdb) list
42 github.com/<username>/<project>/_test/_testmain.go: No such file or directory.
This means I cannot happily inspect the Go source code in GDB like I'm used to. I know it is possible to force the temporary directory to stay by passing the -work flag to the go test command, but then it is still a huge hassle since the binary is not created in that directory and such. I was wondering if anyone found a clean solution to this problem.
Go 1.5 has been released, and there is still no officially sanctioned Go debugger. I haven't had much success using GDB for effectively debugging Go programs or test binaries. However, I have had success using Delve, a non-official debugger that is still undergoing development: https://github.com/derekparker/delve
To run your test code in the debugger, simply install delve:
go get -u github.com/derekparker/delve/cmd/dlv
... and then start the tests in the debugger from within your workspace:
dlv test
From the debugger prompt, you can single-step, set breakpoints, etc.
Give it a whirl!
Unfortunately, this appears to be a known issue that's not going to be fixed. See this discussion:
https://groups.google.com/forum/#!topic/golang-nuts/nIA09gp3eNU
I've seen two solutions to this problem.
1) create a .gdbinit file with a set substitute-path command to
redirect gdb to the actual location of the source. This file could be
generated by the go tool but you'd risk overwriting someone's custom
.gdbinit file and would tie the go tool to gdb which seems like a bad
idea.
2) Replace the source file paths in the executable (which are pointing
to /tmp/...) with the location they reside on disk. This is
straightforward if the real path is shorter then the /tmp/... path.
This would likely require additional support from the compiler /
linker to make this solution more generic.
It spawned this issue on the Go Google Code issue tracker, to which the decision ended up being:
https://code.google.com/p/go/issues/detail?id=2881
This is annoying, but it is the least of many annoying possibilities.
As a rule, the go tool should not be scribbling in the source
directories, which might not even be writable, and it shouldn't be
leaving files elsewhere after it exits. There is next to nothing
interesting in _testmain.go. People testing with gdb can break on
testing.Main instead.
Russ
Status: Unfortunate
So, in short, it sucks, and while you can work around it and GDB a test executable, the development team is unlikely to make it as easy as it could be for you.
I'm still new to the golang game but for what it's worth basic debugging seems to work.
The list command you're trying to work can be used so long as you're already at a breakpoint somewhere in your code. For example:
(gdb) b aws.go:54
Breakpoint 1 at 0x61841: file /Users/mat/gocode/src/github.com/stellar/deliverator/aws/aws.go, line 54.
(gdb) r
Starting program: /Users/mat/gocode/src/github.com/stellar/deliverator/aws/aws.test
[snip: some arnings about BinaryCache]
Breakpoint 1, github.com/stellar/deliverator/aws.imageIsNewer (latest=0xc2081fe2d0, ami=0xc2081fe3c0, ~r2=false)
at /Users/mat/gocode/src/github.com/stellar/deliverator/aws/aws.go:54
54 layout := "2006-01-02T15:04:05.000Z"
(gdb) list
49 func imageIsNewer(latest *ec2.Image, ami *ec2.Image) bool {
50 if latest == nil {
51 return true
52 }
53
54 layout := "2006-01-02T15:04:05.000Z"
55
56 amiCreationTime, amiErr := time.Parse(layout, *ami.CreationDate)
57 if amiErr != nil {
58 panic(amiErr)
This is just after running the following in the aws subdir of my project:
go test -c
gdb aws.test
As an additional caveat, it does seem very selective about where breakpoints can be placed. Seems like it has to be an expression but that conclusion is only via experimentation.
If you're willing to use tools besides GDB, check out godebug. To use it, first install with:
go get github.com/mailgun/godebug
Next, insert a breakpoint somewhere by adding the following statement to your code:
_ = "breakpoint"
Now run your tests with the godebug test command.
godebug test
It supports many of the parameters from the go test command.
-test.bench string
regular expression per path component to select benchmarks to run
-test.benchmem
print memory allocations for benchmarks
-test.benchtime duration
approximate run time for each benchmark (default 1s)
-test.blockprofile string
write a goroutine blocking profile to the named file after execution
-test.blockprofilerate int
if >= 0, calls runtime.SetBlockProfileRate() (default 1)
-test.count n
run tests and benchmarks n times (default 1)
-test.coverprofile string
write a coverage profile to the named file after execution
-test.cpu string
comma-separated list of number of CPUs to use for each test
-test.cpuprofile string
write a cpu profile to the named file during execution
-test.memprofile string
write a memory profile to the named file after execution
-test.memprofilerate int
if >=0, sets runtime.MemProfileRate
-test.outputdir string
directory in which to write profiles
-test.parallel int
maximum test parallelism (default 4)
-test.run string
regular expression to select tests and examples to run
-test.short
run smaller test suite to save time
-test.timeout duration
if positive, sets an aggregate time limit for all tests
-test.trace string
write an execution trace to the named file after execution
-test.v
verbose: print additional output
I have read the following SO question:
Do specific action when certain breakpoint hits in gdb
Here, we use 'command' to decide what to do when the SPECIFIED Breakboint Gets Hit.
My Question is:
Suppose I put Breakpoints on ALL the Functions matching a given pattern:
gdb$rbreak func_
=> 100 Breakpoints (say)
When I execute this Code, I want to do the SAME Action - on hitting Each of these functions.
Hence, I cannot define something like:
command break_point_number
// since I don't know how many breakpoints will be there
Can somebody please suggest me:
How can I do a specific action-set when ANY Breakpoint gets Hit in GDB?
Thanks.
With a new enough version of gdb you can use a range:
(gdb) rbreak whatever
... gdb creates breakpoints N, N+1, ..., M
(gdb) commands N-M
> stuff
> end
I forget exactly when this feature went in.
With an older version of gdb, I'm not sure it can easily be done.
It can be done with difficulty: use set logging to write output to a file, then "info break", then "shell" to run scripts to edit the file into gdb commands, then "source". This is very painful.