Remote GDB with Gem5 - remote-debugging

gem5 has built-in support for GDB’s remote debugger interface. I want to debug a very tiny code using remote gdb with Gem5 (in se mode).
As we can see from the below image, the program I want to debug (just for testing) has a scanf statement to read a value from user. However, when I type cont in the gdb (on host machine) the program (at simulated machine in Gem5) does not stop at this statement to enter a value. Even I set a breakpoint at this statement, also, the program at gem5 does not stop (to enable me to enter the value of int variable) and finishes its work.
As we can see from the gem5's output... the value of int variable is randomly assigned.
what is really the issue here? is it because scanf statement? or I wrong in somethings?!
How to work with remote gdb in tandem with Gem5?
Both the host and the simulated machine are x86 (no need for cross-gdb)
Regardless of scanf, I have a very simple program that contains three print instructions. I put a breakpoint at the third print instruction, both programs at host and Gem5 stop at this point. when I cont the program at host, it continues its work normally, but at gem5 the program gets stuck and does not continue.
Cross-gdb
The above work was conducted using x86 as a host machine with x86-Gem5. Now I want to test cross-gdb. In my host machine I installed all cross-tools required (gcc, g++ and gdb) to work with ARM simulated machine in Gem5.
From the host machine I started the gdb and typed the following commands:
gdb-multiarch
file arm_test // a sample code compiled using arm-gcc cross compiler
set architecture arm
set b main
target remote localhost:7000
c
when I hit enter after typing c (cont) the program at Gem5 does not stop at main (as I set a breakpoint there) and it finishes its work, prints hello world and exits.
At the Gem5 I saw a warn message says: warn: Invalid breakpoint length
Updated
I have used the following GDB flags to figure out and inspect the data being transmitted to the (host gdb) from Gem5 and vice versa.
--debug-flag=GDBSend,GDBRecv,GDBMisc
As we can see from the image below
when I set a breakpoint at a specific location in the code (say function call) that is located at address offset (1031e) ... a bunch of information is send to Gem5 as it's clear from the image:
0: system.remote_gdb: recv: m1031e,2 // receive
0: system.remote_gdb: send: fff7 // send what ???
0: system.remote_gdb: recv: Z0,1031e,3 // receive
I hope someone help me and explain what does "Z0,1031e,3" really mean? I know that the gem5 has received data from (host gdb) which tell it to set a breakpoint at instruction located at offset 1031e. BUT WHAT does the number three mean here?
Because I think it is the main cause of the problem; when I hit cont at (host gdb) the program at gem5 side never stops and finishes its work with the mentioned warn message: warn: Invalid breakpoint length
WHAT is the valid length? and how to adjust it?
if I set a bp at another location (say at a place of printf instruction), also I got an error.
0: system.remote_gdb: set software breakpoint, addr=0x1030a, len=2
warn: Invalid breakpoint length
Any help or suggestions please!
Gem5 branch: Stable
Config file: se.py
thanks

Related

Saving and restarting a paused gdb session

My understanding is that gdb can monitor the complete state of a running program. Can I save a gdb session that is paused at a breakpoint and resume the session later?
My first attempt was simply generating a core dump in a first gdb session that was paused at a breakpoint and then using the core dump to start a second gdb session.
Saving core file in gdb
This resulted in the following error.
Program terminated with signal SIGTRAP, Trace/breakpoint trap.
So breakpoint information is inserted into the program state, interesting. On my second attempt I did the same but this time I added the same breakpoint to the second session as were in the first session.
Getting gdb to save a list of breakpoints?
Still, I get the same error.
Can I save and restart a gdb session? If so, how?
I don't think this is directly relevant but I'm also getting this warning.
warning: core file may not match specified executable file.
Is gdb simply stating that such a thing is possible in general or does gdb believe this may have happened in the running session? I'm confident that the same executable that produced the core dump is being run under gdb.
Edit: For anyone else who comes along, this question: Save a process' memory for later use? adds to Mats Petersson's answer and links to this article: http://blogs.msdn.com/b/oldnewthing/archive/2004/04/20/116749.aspx which is an interesting read. The linked question also has the suggestion of wrapping the process up in a VM.
I doubt that will ever work. Handles for files and any other resources (semaphores, shared memory, serial ports, network connections and lots of other things) that the program has opened/created will be lost when you save the core-file. You can inspect it, but you can't "continue". A core-file is simply a copy of all the memory that original program was using. Anything else is "lost" when the program terminates. In other words, a core-file will only be useful to inspect things later on, but you can't run, step or continue in a core-file debug session. Only "look at things". And if you can't execute, breakpoints won't really work either... ;)

Stack trace issues on GDB

I am using GDB in remote configuration. So I have gdbserver running on a ARM HW running linux, I connect to remote gdbserver from local gdb, I am able to put breakpoints in initial part of code and ensure that basically GDB works. However when I let my SW run for a while and break to see threads, I don't get useful stack traces for threads. All stack frames are hex addressed none resolving to symbols present in my binaries. Most of them also look the same with gdb also hinting with "same stack frames, corrupted stack ??"
Since SW runs fine with or without debugger I don't doubt my SW. Anybody seen this issue or any idea what might be going on here.
Thanks

Issue with GDB, JTAG and CPU32

I am using GDB along with a JTAG device, an Abatron BDI2000, to debug a programs running on a Motorola M68332.
The 68332 does not have any hardware breakpoint registers. It has very primitive debugging features.
The build tools do not generate 'elf' files, so no symbols for GDB to use.
Also the program I'm debugging is running in Flash.
In fact the 68332 has only one debug instruction, ti. ti by itself steps to the next assembly instruction. ti xxx steps until the address xxx is reached. [Yes, this is caveman days, cold hammer and chisel :)]
I am able to use GDB with target remote to connect to the BDI2000 and issue the GDB commands 'nexti'. Due to the limitations of the 68332, 'stepi' is equivalent to 'nexti'.
Single stepping is only command available.
The monitor command 'monitor ti ' states change the program counter to and step.
If one uses a 'monitor' command that changes the registers, then GDB does not know about the command and its register cache become out of sync. I have created GDB functions which have the GDB command 'flushregs' at the end of each of them. This marks the register cache dirty. The GDB command will fetch a new set of registers.
I would like to create a symbol table file for debugging, but have not found any documentation on the GDB symbol file format.
Are there alternatives to what I have setup?
I do have a RAM overlay for the Flash area. Would this allow software breakpoints?
Thanks in advance for any advice.
I found I can use 'convenience' variables as a substitute for symbols, since I'm not using ever symbol in the program all at once.
set $Symbol=(unsigned int*)<address>
Each 'Symbol' is declared a pointer to an unsigned int at an address. One can put these statements in .gdbinit, and add to them over time.
One can then state
break $Symbol
I show a GDB command function that can be passed one of these 'convenience' variables in the question linked below.
How do I write a GDB function to make a comparison to the program counter

How to debug with strace -i when everytime address is different

[b77d0424] open("etc/shadow",0_RDONLY) = -1 EACCESS (Permission denied)
every time i run [b77d0424] changed to another address
i can not use gdb b *0xb77d0424 and then c to find lib64/libc.so.6
it seems not the same mentioned in a linux programming book
after running ubuntu 13.04 in virtual box
every time i run [b77d0424] changed to another address
This is happening because of address space layout randomization, which you can disable with setarch -R command.
GDB also disables address randomization by default, but the chance that the same address you'll get in GDB and under strace is quite small, as the execution environment under the two tools is quite different. You don't actually need to find the address under strace, you can find it in GDB:
catch syscall open
run
You are now looking at one of the open system calls your program does. Use continue until you stop at the one you are interested in. Now use info registers to find the address of the first parameter, and set a watchpoint on that address.

gdb not hitting breakpoints

To learn a bit more about FreeBSD and *nix systems in general, I'm starting to look at the binaries from the DEFCON 17 Capture The Flag game. Right now, I'm reversing the tucod binary. Here's some possibly useful information on tucod:
tucod: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), for FreeBSD 7.2, dynamically linked (uses shared libs), FreeBSD-style, stripped
Some other possibly useful information gained from some brief static analysis is that tucod binds on port 0xDEAD (cute, eh?) and if you give it a specific password ("HANGEMHIGH!") it will play a game of hang-man with you.
The problem that I'm encountering is that I'm not hitting my breakpoints in gdb. Specifically, the breakpoint that I'm trying to reach is in the code that handles the client connection. Without breakpoints, the code executes as expected. When I set a breakpoint on that code, the child exits (instead of breaking into gdb, as expected). If I set breakpoints before the server forks off the child, I can hit those fine but after hitting "continue" the child does not continue to process my connection (that is, it won't ask me for a password or play hang-man).
Since the daemon forks when it receives a new connection, I try to tell gdb to follow the child with this command:
(gdb) set follow-fork-mode child
But after single-stepping the instructions after the fork, it appears that this isn't working.
I've tried looking for calls to signal, thinking they implemented a custom SIGINT handler (or similar), but the only call to signal that I can see handles SIGCHLD.
My breakpoint in gdb currently looks like this:
(gdb) info breakpoints
Num Type Disp Enb Address What
1 breakpoint keep y 0x080497d0
And 0x080497d0 is the address I want to break on in the client processing code.
I'm sort of new to analyzing software on *nix systems and could use some pointers. How else should I go about troubleshooting why GDB will not hit my breakpoints? Or is there something major I'm just overlooking?
There's a torrent available with all of the game binaries for those interested in seeing the binary first-hand.
Look here for the answer. In short, it looks like GDB supports child debug mode only on HP-UX and Linux.