I have a program which create many threads. I can check it using following command: ps -L pid. I also know that a process loads some shared libs. I wonder if is possible to check which threads belong to a selected shared lib.
That process contains debug symbols and I can attach to them using follwoing command: sudo gdb -p pid What's next ?
Let we already attached to a process.
(gdb) info threads
Will display currently known threads. The last column in the output shows function and library for the last stack frame for each thread.
If you want to see threads start routines and the libraries they belong to, you may use:
(gdb) thread apply all bt -3
This command will show you 3 stack frames (from bottom) for each thread. If you are using pthread library then function that goes right after start_thread() is your start routine.
Related
I have read how gdb attaches to another running process from this link:
How does a debugger peek into another process' memory?
But what about starting a program directly with it as such:
gdb ./my_program
Does gdb fork and run my_program using it and attach to it like it's explained int he above link (i.e with ptrace on linux), or the process is entirely different?
or the process is entirely different?
No: the process is the same -- fork, parent calls PTRACE_ATTACH, child calls PTRACE_TRACEME. That last point guarantees that GDB can debug the child process from its first instruction.
There is one additional complication: GDB uses $SHELL in order to handle input / output redirection, so there is fork -> exec shell (in the child) -> exec program.
Just want to ask is there any method to check if a 3rd API will create a new thread for c/c++ program in linux? As following, assume do_something_API is a 3rd API and we don't know the implementation, then how to know if the funciton will create a new thread? Use gdb or other tools?
int main() {
...
//call 3rd party API
do_something_API();
...
}
how to know if the funciton will create a new thread?
Just stop at do_something_API() line in main function in gdb and use next command once to execute do_something_API() function. If that function creates any new threads, you will see messages from gdb like:
[New Thread 0x41e02940 (LWP 25582)]
See in documentation:
Whenever GDB detects a new thread in your program, it displays the
target system’s identification for the thread with a message in the
form ‘[New systag]’, where systag is a thread identifier whose form
varies depending on the particular system. For example, on GNU/Linux,
you might see
[New Thread 0x41e02940 (LWP 25582)]
how to know if the funciton will create a new thread?
You may have an XY problem. What are you actually trying to achieve?
Read the documentation or ask 3rd party developer. If they promise to never create threads, then that's the answer. Otherwise, assume that they may (if not in the current version, then in the next one).
You can run nm libsomething.{a,so} | grep pthread_create and strings libsomething.{a,so} | grep pthread_create. If both commands produce no output, you can be pretty sure that the current version of the library will not create new threads.
If you run the test program under GDB, and next over the do_something_API() call, GDB will report new thread creation with messages similar to [New thread ...]. If you don't see such messages, no new thread was created.
You could also set a breakpoint on pthread_create, or use info thread before and after the call.
Note: if no new threads are created, this is a very weak indicator: do_something_API() may decide whether or not to create new threads depending on runtime environment (e.g. an environment variable, or current directory, or time of day), and so the next time you run the test the answer may change.
you can try running your code in gdb and use "info threads" to see the all running threads within your program.
or you can also check using /proc/
got some big real time project to deal with (multiple processes (IPCs), multi Everything in short).
My working on process is started as service on Linux. I have the root access.
Here is the problem:
I'm trying to attach to a running proc, tried starting it through/with gdb but the result is the same: it stops the executable once I "touched" it with gdb or sometimes it throws:
Program received signal SIGUSR1, User defined signal 1. [Switching to Thread 0x7f9fe869f700 (LWP 2638)]
of course from there nothing can be done.
Tried:
handle all nostop
attach to launched as service (daemon) or launched as regular proc
started from gdb
thought maybe forking/multi-threaded problem - implemented in the very beginning sleep for 10 seconds - attached to it with "continue"
Guys, all I want it is to debug, hit the breakpoints, etc.
Please help! Share ideas.
Editing actual commands:
1) gdb attach myProcId. Then after reading symbols, I hit "c" which results:
Program received signal SIGUSR1, User defined signal 1.
[Switching to Thread 0x7f9fe869f700 (LWP 2638)]
0x00007f9fec09bf73 in select () from /lib64/libc.so.6
2) If I make the first line 10 seconds sleep in the code, attaching to the process, hit "c", result: it runs, shows info threads, backtrace of main, but never hits the breakpoint (for sure the code runs there - I get logs and different behaviour if I change code there), meaning the process is stuck.
3) All other combinations like gdb path/to/my/proc args list, then start. Where arg list played with different related options gdb gives us.
Maybe worth to mention: process network packets related, timers driven also.
But for me the important thing is a current snapshot on break, i don't care what will happen to the system after timers expired.
Since you mentioned that you are debugging a multiprocessing program, I think the underlying program you have is to set the breakpoint in the correct subprocess.
Try break fork and set follow-fork-mode child/parent. What you want to achieve is have gdb attached to the process that is running the code you want to debug.
Refer to this link.
Another thought is to generate a crash, since you can compile the programe. For example add a int i = *(int*)NULL and that will generate a core dump. You can then debug the core dump with gdb <program> <core dump>. You can refer to this page for how to configure core dump.
If I have some multithreaded process and want to trace it with gdb using attach command, to which thread it will connect (e.g. current running or main)? I know that I can discover it with info threads but I want to know which thread it will choose by default.
For Linux, all of the threads are stopped by the ptrace command when gdb attaches.
It has been my experience that gdb defaults to the main thread for C/C++ applications. If you attach to a process and do a 'bt' it will list the stack for 'main'.
Information is available for all threads however. gdb can look at the thread(s) information in the /proc filesystem. The proc contains information about each thread in the tasks area. Details about the stack address is located in the stat file as well as the maps file. Details are also available regarding the register values for each thread.
Along the lines of your question, I've often wondered why stepping through a multithreaded application will cause gdb to jump from thread to thread. I think that gdb is still at the mercy of the kernel scheduler so that a step on a thread may lead to a different thread getting the CPU resource and a breakpoint being triggered.
On Linux, where thread ids exist in the same space as process ids, it appears you can run gdb -p tid to attach to the thread with given tid and its owning process, without knowing the pid. Because the main thread of a process has tid == pid, it makes sense that running gdb -p pid connects to the main thread.
Example code that connects gdb to the currently executing thread, e.g. for generating a pretty stack trace: https://github.com/facebook/rocksdb/pull/11150
Without getting into to to much detail, I'm working on a program that consists of several separate processes all running on embedded QNX RTOS. They don't have a parent-child relationship, they are all spawned using spawnlp(P_NOWAIT, ...) and they all communicate with each other using the IPC mechanism provided by the OS.
When I'm debugging with GDB and I hit a breakpoint in the process I'm working in, all of my threads are paused, which is great. But is there a way to also have it pause execution of my other processes? Right now what's happening is all the other processes keep on truckin' while my process is paused and so all the IPC queues get full etc. etc.
Thanks in advance,
HF
You can associate a list of GDB commands with each breakpoint. So when you hit a breakpoint in process A, you can for example send a SIGTRAP to process B, which should drop it into the debugger:
(gdb) b main
Breakpoint 1 at 0x804834a: file testA.c, line 40.
(gdb) command
Type commands for when breakpoint 1 is hit, one per line.
End with a line saying just "end".
>shell kill -s TRAP `pidof testB`
>end
(gdb)
More info at Breakpoint Command Lists